[syslog-ng] Reload does not free up RAM

[Evan Rempel]

Balazs Scheidler wrote:
> On Wed, 2008-10-01 at 09:54 -0700, Evan Rempel wrote:
>> We have cases where a syslog-ng buffers quite a bit of data (2GB or so) before
>> it is able to flush the data to the destination. Currently, the only way to "free up" that RAM
>> is to restart syslog-ng, which means that some log messages are lost during the restart
>> window.
>>
>> I open for discussion the idea that on a reload that syslog-ng flush and free all buffer
>> space and essentially start from a "just started" state again.
> 
> Well, this is currently a PE feature, and BalaBit decided not to open
> source it this time with 3.0 as that's one of the primary incentives to
> buy the commercial version.
> 
> PE with its persistent disk buffers is saving queue contents upon
> restart right into the disk buffer file. And also, syslog-ng does not
> buffer so much data in memory as it first uses the disk.

The disk buffering offers a lot more than this.

Persistent buffers around a syslog restart
Persistent buffers around a system restart (think power outage)
Much larger buffering (potentially hundreds of GBytes) with minimal
system impact.

I would hope that PE version would shrink its disk buffers once they
are flushed, just like I am hoping that OSE would shrink its RAM buffers
once they are flushed.

I don't think that RAM usage AFTER buffers are flushed is much of an
incentive to purchase the PE. The persistence and size of the buffers
is the incentive.

Thanks for the background.

-- 
Evan Rempel