[syslog-ng] syslog-ng memory usage grows
ihrwein at gmail.com
Wed Dec 17 23:42:05 CET 2014
I checked your valgrind log and the redis module and I think the leak is in
This returns a reply object, but it's ignored and never freed.
If we make a patch on GitHub, could you test it?
2014-12-17 18:36 GMT+01:00 <jrhendri at roadrunner.com>:
> Hi Fabien,
> Hope the season is treating you well.
> I ran syslog-ng for 10 minutes (recall, ~7000 EPS incoming) under
> valgrind and here is the output (attached).
> It looks like there is some sort of leak as the process from yesterday
> had grown to over 16GB before I stopped it for the valgrind test.
> Thanks for the help (as always).
> And please let me know if I can provide any other data.
> ---- Fabien Wernli <wernli at in2p3.fr> wrote:
> > Hi Jim,
> > On Tue, Dec 16, 2014 at 01:24:28PM -0500, Jim Hendrick wrote:
> > > I'm not sure what I am asking, other than general advice on:
> > > - performance using patterndb
> > performance is awesome, I wouldn't worry about it
> > although it would help to have some counters on the parsers in the stats
> > interface. That bieng said, it could help you to increase the stats
> > > - performance using redis destination
> > I can't comment on that one, sorry
> > > - advice on debugging where this memory growth is happening
> > Could you run syslog-ng inside `valgrind --tool=memcheck
> > --trace-children=yes --leak-check=yes`?
> > The problem is this will really slow down your instance, but let it run a
> > few minutes nevertheless and pastebin the result somewhere.
> > > As a rough measure - I have a syslog-ng process that has been running
> > > less than 3 hours and right now is using 1.52GB of resident memory
> (shown by
> > > "top")
> > I'm suspecting `format-json` to be the source of the leak, or the flow
> > control cache. These have been the leak source of our own observations
> > times, although many patches have solved the issues.
> > > (I would rather parse things in syslog-ng, but I *could* do this all
> > > logstash/grok if this proves too much for patterndb at this load)
> > if you're worried about performance, I wouldn't even consider to begin
> thinking about comparing
> > LS/grok to patterndb
> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Documentation:
> > FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the syslog-ng