[syslog-ng] Heavy memory leak using correlation or patterndb

[Balazs Scheidler]

On Sat, 2011-02-05 at 15:47 +0100, Balazs Scheidler wrote:
> On Wed, 2011-01-26 at 11:01 +0100, Guillaume Rousse wrote:
> > Hello.
> > 
> > Since I installed the rules dedicated to dropping LDAP probe connection
> > traces, the memory usage of my ldap server exploded. top confirms than
> > syslog-ng is the culprit, using 60% of the available memory. Here is a
> > weekly munin graph presenting the issue:
> > http://www.zarb.org/~guillomovitch/memory-week.png
> > 
> > The rules have been installed on 19th, and I restarted syslog-ng yesterday.
> > 
> > I'm using syslog-ng 3.2.1, on mandriva 2010.0, 64 bits. Here are the
> > exact rules used, if that matters (I couldn't try latest Balazs
> > suggestions yet). The context-timeout for the first rule is quite low,
> > because the probe checks every two seconds. By any chance, doesn't the
> > second timeout, set to 0, set an illimited session expiration time ?
> > 
> > If not the case, I'd gladly try any needed experiment, such as running
> > syslog-ng under valgrind, to identify the leak source.
> 
> Hmm.. thanks for the report, and sorry for not going after it any
> sooner. I'll try to reproduce the problem as no immediate culprit is
> visible without reproduction. I hope to get it fixed right after I send
> this email, so that hopefully means today.
> 
> If not, then I'm not sure when I'll have a chance to work on it
> again. :(
> 

I wasn't able to reproduce it either with current git HEAD (3.2.2 +
unrelated fixes) or v3.2.1.

I was using your example messages to produce a file with 20000 log
messages and then told pdbtool to read the file using pdbtool match, and
then even fed the messages to syslog-ng using a db-parser() with your
patterndb. Neither cases did it start to grow.

I've checked using valgrind, but nothing suspicios. So your setup is
definitely affecting it somehow.

Can you please try to reproduce it with pdbtool match first (that would
be better and easier to diagnose) with a concrete logfile that is
processed by your patterndb rules?

If that doesn't start growing, then the only chance is to catch
syslog-ng while it is growing using valgrind. A preferred command line
to do that would be:

G_SLICE=always-malloc valgrind --leak-check=yes --show-reachable=yes /sbin/syslog-ng ...

Thanks for your efforts for helping me to track this down.

-- 
Bazsi