[syslog-ng] 3.3.4 anyone else notice a memory leak

[Jakub Jankowski]

On 2012-02-10, Evan Rempel wrote:

>>> I am fairly certain that there is a memory leak. How do I help track
>>> it down?
[...]
> Yup, I can reproduce it. The output of valgrind does not appear to
> show the leak. If I run syslog-ng and do NOT send any data to it (it 
> processes no
> log lines) the output of valgrind is the same as if it processes 100,000 log 
> lines,
> however, when I "dump" 100,000 log lines
>
> % wc /home1l/erempel/log
>  107590  1233928 18956930 /home1l/erempel/log
>
> into the pipe (cat /home1l/erempel/log > pipe) the memory footprint of 
> syslog-ng
> grows by 3MB, every time I dump in the log lines.

Okay, this might be a long shot, but... can you try to revert commit [1], 
recompile syslog-ng and reproduce the leak? (grab the tarball, unpack it, 
get the raw form of [1], and use -R option to reverse the patch; then 
your usual build procedure)

This commit was applied right after 3.3.2 was released, because it 
supposedly fixed crash with tcp destinations. Bazsi had doubts[2] that it 
may reintroduce a memory leak... and now I know it *does* reintroduce the 
leak, because it was leak I originally reported (longish thread here, see 
[3], [4], [5]) and now I'm seeing it again, with both 3.3.3 and 3.3.4:

==3645== LEAK SUMMARY:
==3645==    definitely lost: 3,719,421 bytes in 10,008 blocks
==3645==    indirectly lost: 905,917 bytes in 10,014 blocks
==3645==      possibly lost: 17,214 bytes in 128 blocks
==3645==    still reachable: 80,063 bytes in 3,278 blocks
==3645==         suppressed: 0 bytes in 0 blocks


I've just re-ran my tests[6] with commit [1] reverted, and the results are 
good again:

==7885== LEAK SUMMARY:
==7885==    definitely lost: 297 bytes in 6 blocks
==7885==    indirectly lost: 1,025 bytes in 22 blocks
==7885==      possibly lost: 13,527 bytes in 110 blocks
==7885==    still reachable: 106,235 bytes in 3,327 blocks
==7885==         suppressed: 0 bytes in 0 blocks


I understand that my use-case scenario is a bit different than yours, but 
perhaps the underlying issue is the same?

Anyway - Bazsi, Gergely, could you please take a look at this whole thing? 
Because looks like we either end up with huge memleaks (with [1]) or 
tcp()-related crashes (without it).

Hope this helps a bit.

[1] http://git.balabit.hu/?p=bazsi/syslog-ng-3.3.git;a=commitdiff;h=c7070e2a6f1c3a312260bcecf49d62028fef27ce
[2] https://lists.balabit.hu/pipermail/syslog-ng/2011-November/017697.html
[3] https://lists.balabit.hu/pipermail/syslog-ng/2011-September/017269.html
[4] https://lists.balabit.hu/pipermail/syslog-ng/2011-September/017275.html
[5] https://lists.balabit.hu/pipermail/syslog-ng/2011-September/017338.html
[6] https://lists.balabit.hu/pipermail/syslog-ng/2011-October/017555.html


Regards,
  Jakub

-- 
Jakub Jankowski|shasta at toxcorp.com|http://toxcorp.com/
GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D