[syslog-ng] Profiling syslog-ng memory usage and performance

[Balazs Scheidler]

Hi William,

On Tue, 2012-02-21 at 14:26 -0800, William Sweat wrote:
> Hello,
> 
> I’m having issues with syslog-ng hitting swap on a set of servers that
> handle a lot of web traffic.  Are there good techniques to figure out
> an optimal configuration or if there’s something with syslog-ng that
> may prove problematic with large amounts of requests?
> 
> I’m using the syslog-ng premium client (LTS v4.0.3b), on Linux systems
> with 256GB of Memory and RAID10.  I’ve enabled more informational
> logging, but because each system handles multiple gigabytes of traffic
> per day, debugging is problematic as this is a problem that manifests
> after a day (or three).  Also I am using nightly logrotate, so
> syslog-ng does get restarted everyday to write new log files.  These
> are only high traffic servers that are experiencing swap issues, the
> syslog-ng server doesn’t have any problems.

First of all, the easiest way to get support for PE is to ask on the
official support channel, especially if you think you've found a bug.
The OSE and PE codebases are related (PE is using OSE at its heart), but
it takes a while while bugfixes propagate to the PE tree from here.

Regarding your question, you might have hit a memory leak somewhere, and
I know that such things were fixed in the just released 4.0.5.

Here's the changelog for that:

http://www.balabit.com/files/syslog-ng/premium-edition/4.0.5/changelog-en.txt

And here's the bugfix that I was referring to:

#24546: Memory leak when reading from disk buffer

This may affect you, if you are using syslog-ng as a client (which I
think you do), and you've enabled the disk buffer feature.

-- 
Bazsi