https://bugzilla.balabit.com/show_bug.cgi?id=257 Gergely Nagy <algernon@balabit.hu> changed: What |Removed |Added ---------------------------------------------------------------------------- Status|NEW |ASSIGNED --- Comment #1 from Gergely Nagy <algernon@balabit.hu> 2013-11-01 13:50:22 --- Do you perhaps have a rough idea when (as in which version of syslog-ng) this leak started to happen? -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

https://bugzilla.balabit.com/show_bug.cgi?id=257 --- Comment #3 from Evan Rempel <erempel@uvic.ca> 2013-11-01 21:47:01 --- OK. I can easily produce this on my test system with NO messages going to it. valgrind reports [root@pangolin log]# cat valgrind.out ==2726== Memcheck, a memory error detector ==2726== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al. ==2726== Using Valgrind-3.8.1 and LibVEX; rerun with -h for copyright info ==2726== Command: /usr/local/sbin/syslog-ng --cfgfile=/usr/local/etc/syslog-ng/syslog-ng.server.conf --persist-file=/var/local/syslog-ng.server.persist --pidfile=/var/run/syslog-ng.server.pid --control=/var/local/syslog-ng.server.ctl --process-mode background ==2726== Parent PID: 16184 ==2726== ==2726== Warning: noted but unhandled ioctl 0x5422 with no size/direction hints ==2726== This could cause spurious value errors to appear. ==2726== See README_MISSING_SYSCALL_OR_IOCTL for guidance on writing a proper wrapper. ==2726== ==2726== HEAP SUMMARY: ==2726== in use at exit: 16,335 bytes in 51 blocks ==2726== total heap usage: 104 allocs, 53 frees, 70,285 bytes allocated ==2726== ==2726== LEAK SUMMARY: ==2726== definitely lost: 216 bytes in 1 blocks ==2726== indirectly lost: 1,991 bytes in 26 blocks ==2726== possibly lost: 2,054 bytes in 8 blocks ==2726== still reachable: 12,074 bytes in 16 blocks ==2726== suppressed: 0 bytes in 0 blocks ==2726== Rerun with --leak-check=full to see details of leaked memory ==2726== ==2726== For counts of detected and suppressed errors, rerun with: -v ==2726== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 6 from 6) Did you want binaries, core dump, recompile with -g ... just let me know. I have added an attachment with the --leak-check=full output -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

https://bugzilla.balabit.com/show_bug.cgi?id=257 --- Comment #5 from Gergely Nagy <algernon@balabit.hu> 2013-11-02 00:30:37 --- Can you do a valgrind run with --leak-check=full --show-reachable=yes --verbose (preferably on a syslog-ng compiled with -ggdb3 -O3 (no need for -O0 in this case)), and also do about a handful or more reloads so the leak becomes more easily visible? -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

https://bugzilla.balabit.com/show_bug.cgi?id=257 --- Comment #7 from Gergely Nagy <algernon@balabit.hu> 2013-11-02 18:40:37 --- (In reply to comment #6)

Created an attachment (id=85) --> (https://bugzilla.balabit.com/attachment.cgi?id=85) [details] valgrind --leak-check=full --show-reachable=yes --verbose

here is the valgrind --leak-check=full --show-reachable=yes --verbose

on a syslog-ng compiled with -ggdb3 -O3

Hrm. How many reloads was this? Valgrind only reports about 16-17k leaked memory which isn't all that much... though... syslog-ng is using the slice allocator, which tends to confuse valgrind at times. I'm sorry I forgot this earlier, but would it be possible to re-run the test with G_SLICE=always-malloc set in the environment too? And the more reloads the better. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

https://bugzilla.balabit.com/show_bug.cgi?id=257 --- Comment #9 from Evan Rempel <erempel@uvic.ca> 2013-11-02 19:47:18 --- Created an attachment (id=86) --> (https://bugzilla.balabit.com/attachment.cgi?id=86) valgrind --leak-check=full --show-reachable=yes --verbose with G_SLICE=always-malloc here is the next one. 16 reloads ps auxwww went from USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND root 28627 9.4 8.3 223564 84956 ? Ss 11:36 0:03 valgrind --log-file=valgrind-full-verbose.txt .... root 28627 4.7 22.7 372640 232628 ? Ssl 11:36 0:14 valgrind --log-file=valgrind-full-verbose.txt .... which shows memory going from 223564 KB to 372640 KB which is well over 100 MB. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

https://bugzilla.balabit.com/show_bug.cgi?id=257 --- Comment #11 from Balazs Scheidler <bazsi@balabit.hu> 2013-11-02 20:24:43 --- never forget that valgrind itself eats a _lot_ of memory. but let's see the dump. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

https://bugzilla.balabit.com/show_bug.cgi?id=257 --- Comment #13 from Evan Rempel <erempel@uvic.ca> 2013-11-02 23:47:02 --- The first couple of reloads the footprint grows a little (1M, then 200K) but after that it is stable, at least on my test system. Thanks for the fast turn around. I will place this build into production on Monday and report again. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

https://bugzilla.balabit.com/show_bug.cgi?id=257 --- Comment #15 from Evan Rempel <erempel@uvic.ca> 2013-11-04 20:26:22 --- (In reply to comment #13)

The first couple of reloads the footprint grows a little (1M, then 200K) but after that it is stable, at least on my test system.

Thanks for the fast turn around.

I will place this build into production on Monday and report again.

Same successful behavior on a fully loaded syslog process. The first 2-3 reloads the memory grows a little, and then it becomes stable. I don't think that is any surprise since the 3.4.5 was released yesterday :-) Thanks again for all of the time Balabit puts into the OSE for us all. Evan. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.