[syslog-ng] 3.3.4 anyone else notice a memory leak

Hendrik Visage hvjunk at gmail.com
Thu Feb 9 18:44:17 CET 2012 

Advice from Gergely Nagy in a previous case:

<quote Gergely>
Hrm. Well, valgrind would be best, but that slows things down to almost
a grounding halt. (But in case you want to try that:
G_SLICE=always-malloc valgrind --leak-check=full -v --show-reachable-yes
--log-file=valgrind.log syslog-ng -F would be the command to run ;)

Other than that, a config could be useful, so I can try and reproduce
the problem (or at least identify possible areas where the leak might be
coming from, and go from there), and see if I can fix it.
</quote>

On Thu, Feb 9, 2012 at 6:35 PM, Evan Rempel <erempel at uvic.ca> wrote:
> Hmm. Mine is build from the tar ball as well.
>
> I am fairly certain that there is a memory leak. How do I help track it down?
>
>
> My syslog-ng statistic line shows that nothing was dropped (implying that not much was queued) and that nothing was stored (does
> that mean than nothing ever needed to be queued?).
>
> 2012-02-09T08:34:14-08:00 flexfilter_alert at linnet.comp.uvic.ca/linnet.comp.uvic.ca syslog.info flexfilter_alert: Log statistics; dropped='dst.tcp(localsyslog#0,localhost:1514)=0', processed='dst.tcp(localsyslog#0,localhost:1514)=78', stored='dst.tcp(localsyslog#0,localhost:1514)=0', processed='src.internal(int#0)=78', stamp='src.internal(int#0)=1328804653', processed='destination(localsyslog)=78', processed='source(local)=45390409', processed='destination(flexfilter)=45390409', processed='global(payload_reallocs)=53024', processed='global(msg_clones)=0', processed='source(int)=78', processed='center(queued)=0', processed='global(sdata_updates)=0', processed='center(received)=0', dropped='dst.program(flexfilter#0,/home1l/sysprog/flexfilter/flexfilter /home1l/sysprog/syslog-apps/alert/fullsyslog.xml)=0', processed='dst.program(flexfilter#0,/home1l/sysprog/flexfilter/flexfilter /home1l/sysprog/syslog-apps/alert/fullsyslog.xml)=45390409', stored='dst.program(flexfilter#0,/home1l/sysprog/flexfilter/flexfilter /home1l/sysprog/syslog-apps/alert/fullsyslog.xml)=0'
>
>
> The memory (virtual size) shows
>
> DATE  TIME USER       PID  PPID  NI PRI CPU    VSZ     ELAPSED     TIME COMMAND
> 20120209 000101 root 11545 11544 0 19 - 601332 04:17:01 00:00:57 /usr/local/sbin/syslog-ng
> 20120209 001601 root 11545 11544 0 19 - 632528 04:32:01 00:01:00 /usr/local/sbin/syslog-ng
> 20120209 003101 root 11545 11544 0 19 - 660760 04:47:01 00:01:04 /usr/local/sbin/syslog-ng
> 20120209 004601 root 11545 11544 0 19 - 684580 05:02:01 00:01:07 /usr/local/sbin/syslog-ng
> 20120209 010101 root 11545 11544 0 19 - 707024 05:17:01 00:01:09 /usr/local/sbin/syslog-ng
> 20120209 011601 root 11545 11544 0 19 - 729292 05:32:01 00:01:11 /usr/local/sbin/syslog-ng
> 20120209 013101 root 11545 11544 0 19 - 752112 05:47:01 00:01:14 /usr/local/sbin/syslog-ng
> 20120209 014601 root 11545 11544 0 19 - 773556 06:02:01 00:01:16 /usr/local/sbin/syslog-ng
> 20120209 020101 root 11545 11544 0 19 - 797420 06:17:01 00:01:19 /usr/local/sbin/syslog-ng
> 20120209 021601 root 11545 11544 0 19 - 825760 06:32:01 00:01:22 /usr/local/sbin/syslog-ng
> 20120209 023101 root 11545 11544 0 19 - 847680 06:47:01 00:01:25 /usr/local/sbin/syslog-ng
> 20120209 024601 root 11545 11544 0 19 - 872712 07:02:01 00:01:28 /usr/local/sbin/syslog-ng
> 20120209 030101 root 11545 11544 0 19 - 898816 07:17:01 00:01:31 /usr/local/sbin/syslog-ng
> 20120209 031601 root 11545 11544 0 19 - 925232 07:32:01 00:01:34 /usr/local/sbin/syslog-ng
> 20120209 033101 root 11545 11544 0 19 - 949924 07:47:01 00:01:37 /usr/local/sbin/syslog-ng
> 20120209 034601 root 11545 11544 0 19 - 976864 08:02:01 00:01:40 /usr/local/sbin/syslog-ng
> 20120209 040101 root 11545 11544 0 19 - 1003048 08:17:01 00:01:43 /usr/local/sbin/syslog-ng
> 20120209 041601 root 11545 11544 0 19 - 1027700 08:32:01 00:01:46 /usr/local/sbin/syslog-ng
> 20120209 043101 root 11545 11544 0 19 - 1054552 08:47:01 00:01:49 /usr/local/sbin/syslog-ng
> 20120209 044601 root 11545 11544 0 19 - 1080776 09:02:01 00:01:52 /usr/local/sbin/syslog-ng
> 20120209 050101 root 11545 11544 0 19 - 1105228 09:17:01 00:01:55 /usr/local/sbin/syslog-ng
> 20120209 051601 root 11545 11544 0 19 - 1132676 09:32:01 00:01:58 /usr/local/sbin/syslog-ng
> 20120209 053101 root 11545 11544 0 19 - 1159172 09:47:01 00:02:01 /usr/local/sbin/syslog-ng
> 20120209 054601 root 11545 11544 0 19 - 1185140 10:02:01 00:02:05 /usr/local/sbin/syslog-ng
> 20120209 060101 root 11545 11544 0 19 - 1212596 10:17:01 00:02:08 /usr/local/sbin/syslog-ng
> 20120209 061601 root 11545 11544 0 19 - 1244612 10:32:01 00:02:12 /usr/local/sbin/syslog-ng
> 20120209 063101 root 11545 11544 0 19 - 1278132 10:47:01 00:02:16 /usr/local/sbin/syslog-ng
> 20120209 064601 root 11545 11544 0 19 - 1312416 11:02:01 00:02:20 /usr/local/sbin/syslog-ng
> 20120209 070101 root 11545 11544 0 19 - 1343452 11:17:01 00:02:24 /usr/local/sbin/syslog-ng
> 20120209 071601 root 11545 11544 0 19 - 1377180 11:32:01 00:02:28 /usr/local/sbin/syslog-ng
> 20120209 073101 root 11545 11544 0 19 - 1408764 11:47:01 00:02:32 /usr/local/sbin/syslog-ng
> 20120209 074602 root 11545 11544 0 19 - 1438876 12:02:02 00:02:35 /usr/local/sbin/syslog-ng
> 20120209 080101 root 11545 11544 0 19 - 1475788 12:17:01 00:02:40 /usr/local/sbin/syslog-ng
> 20120209 081601 root 11545 11544 0 19 - 1511372 12:32:01 00:02:44 /usr/local/sbin/syslog-ng
> 20120209 083101 root 11545 11544 0 19 - 1554716 12:47:01 00:02:49 /usr/local/sbin/syslog-ng
>
>
> So you can see that the memory footprint grows constantly.
>
> Evan.
> ________________________________________
> From: syslog-ng-bounces at lists.balabit.hu [syslog-ng-bounces at lists.balabit.hu] On Behalf Of Martin Holste [mcholste at gmail.com]
> Sent: Wednesday, February 08, 2012 8:58 AM
> To: Syslog-ng users' and developers' mailing list
> Subject: Re: [syslog-ng] 3.3.4 anyone else notice a memory leak
>
> Hm, I'm not seeing the leak in ELSA (which uses program()) and I'm
> using 3.3.4 from the latest tarball (not git).
>
> On Tue, Feb 7, 2012 at 11:49 PM, Evan Rempel <erempel at uvic.ca> wrote:
>> It looks like it must be writing to a program destination, or writing to a pipe that leaks the memory.
>>
>> The only syslog-ng processes I have that do not leak memory are;
>>
>> - read from a pipe and writes to files.
>> - read from ksm /dev/log, write to files and tcp destination
>>
>> The rest leak;
>>
>> - Read from network, write to many pipes
>> - Read from pipe and write to program
>>
>> Evan.
>> ________________________________________
>> From: syslog-ng-bounces at lists.balabit.hu [syslog-ng-bounces at lists.balabit.hu] On Behalf Of Evan Rempel [erempel at uvic.ca]
>> Sent: Tuesday, February 07, 2012 9:35 PM
>> To: Syslog-ng users' and developers' mailing list
>> Subject: [syslog-ng] 3.3.4 anyone else notice a memory leak
>>
>> I run a number of syslog-ng processes, and I am noticing that the processes that read from
>> a pipe, and write to a program destination all seem to have a memory leak in them.
>>
>> The rate of leak seems  to be dependent on the volume of log lines (not surprising).
>>
>> Has anyone else noticed this.
>>
>> My config is;
>>
>>
>>
>>
>> @version: 3.3
>>
>> #
>> # UVic syslog-ng configuration
>>
>> options {
>>        log_fifo_size(100000);
>>        use_fqdn(yes);
>>        keep_hostname(yes);
>>        chain_hostnames(yes);
>>        time_reap(60);
>>        time_reopen(5);
>>        flush_lines(1000);
>>        flush_timeout(1000);
>> };
>>
>> source local { pipe("/var/log/syslog.pipes/syslogstats" log_iw_size(90000) log_fetch_limit(500) flags(no-parse)); };
>> source int { internal(); };
>>
>> destination localsyslog { tcp("localhost" port(1514) localip(localhost) log_fifo_size(50000) template("<$PRI>$S_DATE syslogstats@$HOST syslogstats: $MSGONLY\n") template_escape(no) ); };
>>
>> destination syslog_stats { program("/home/sysprog/syslog-apps/syslog_stats/syslogstats" template("$MESSAGE\n") template_escape(no) ); };
>>
>> log { source(local); destination(syslog_stats); flags(flow-control); };
>> log { source(int); destination(localsyslog); };
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>

More information about the syslog-ng mailing list