[syslog-ng] 3.3.4 anyone else notice a memory leak
Evan Rempel
erempel at uvic.ca
Thu Feb 9 17:35:08 CET 2012
Hmm. Mine is build from the tar ball as well.
I am fairly certain that there is a memory leak. How do I help track it down?
My syslog-ng statistic line shows that nothing was dropped (implying that not much was queued) and that nothing was stored (does
that mean than nothing ever needed to be queued?).
2012-02-09T08:34:14-08:00 flexfilter_alert at linnet.comp.uvic.ca/linnet.comp.uvic.ca syslog.info flexfilter_alert: Log statistics; dropped='dst.tcp(localsyslog#0,localhost:1514)=0', processed='dst.tcp(localsyslog#0,localhost:1514)=78', stored='dst.tcp(localsyslog#0,localhost:1514)=0', processed='src.internal(int#0)=78', stamp='src.internal(int#0)=1328804653', processed='destination(localsyslog)=78', processed='source(local)=45390409', processed='destination(flexfilter)=45390409', processed='global(payload_reallocs)=53024', processed='global(msg_clones)=0', processed='source(int)=78', processed='center(queued)=0', processed='global(sdata_updates)=0', processed='center(received)=0', dropped='dst.program(flexfilter#0,/home1l/sysprog/flexfilter/flexfilter /home1l/sysprog/syslog-apps/alert/fullsyslog.xml)=0', processed='dst.program(flexfilter#0,/home1l/sysprog/flexfilter/flexfilter /home1l/sysprog/syslog-apps/alert/fullsyslog.xml)=45390409', stored='dst.program(flexfilter#0,/home1l/sysprog/flexfilter/flexfilter /home1l/sysprog/syslog-apps/alert/fullsyslog.xml)=0'
The memory (virtual size) shows
DATE TIME USER PID PPID NI PRI CPU VSZ ELAPSED TIME COMMAND
20120209 000101 root 11545 11544 0 19 - 601332 04:17:01 00:00:57 /usr/local/sbin/syslog-ng
20120209 001601 root 11545 11544 0 19 - 632528 04:32:01 00:01:00 /usr/local/sbin/syslog-ng
20120209 003101 root 11545 11544 0 19 - 660760 04:47:01 00:01:04 /usr/local/sbin/syslog-ng
20120209 004601 root 11545 11544 0 19 - 684580 05:02:01 00:01:07 /usr/local/sbin/syslog-ng
20120209 010101 root 11545 11544 0 19 - 707024 05:17:01 00:01:09 /usr/local/sbin/syslog-ng
20120209 011601 root 11545 11544 0 19 - 729292 05:32:01 00:01:11 /usr/local/sbin/syslog-ng
20120209 013101 root 11545 11544 0 19 - 752112 05:47:01 00:01:14 /usr/local/sbin/syslog-ng
20120209 014601 root 11545 11544 0 19 - 773556 06:02:01 00:01:16 /usr/local/sbin/syslog-ng
20120209 020101 root 11545 11544 0 19 - 797420 06:17:01 00:01:19 /usr/local/sbin/syslog-ng
20120209 021601 root 11545 11544 0 19 - 825760 06:32:01 00:01:22 /usr/local/sbin/syslog-ng
20120209 023101 root 11545 11544 0 19 - 847680 06:47:01 00:01:25 /usr/local/sbin/syslog-ng
20120209 024601 root 11545 11544 0 19 - 872712 07:02:01 00:01:28 /usr/local/sbin/syslog-ng
20120209 030101 root 11545 11544 0 19 - 898816 07:17:01 00:01:31 /usr/local/sbin/syslog-ng
20120209 031601 root 11545 11544 0 19 - 925232 07:32:01 00:01:34 /usr/local/sbin/syslog-ng
20120209 033101 root 11545 11544 0 19 - 949924 07:47:01 00:01:37 /usr/local/sbin/syslog-ng
20120209 034601 root 11545 11544 0 19 - 976864 08:02:01 00:01:40 /usr/local/sbin/syslog-ng
20120209 040101 root 11545 11544 0 19 - 1003048 08:17:01 00:01:43 /usr/local/sbin/syslog-ng
20120209 041601 root 11545 11544 0 19 - 1027700 08:32:01 00:01:46 /usr/local/sbin/syslog-ng
20120209 043101 root 11545 11544 0 19 - 1054552 08:47:01 00:01:49 /usr/local/sbin/syslog-ng
20120209 044601 root 11545 11544 0 19 - 1080776 09:02:01 00:01:52 /usr/local/sbin/syslog-ng
20120209 050101 root 11545 11544 0 19 - 1105228 09:17:01 00:01:55 /usr/local/sbin/syslog-ng
20120209 051601 root 11545 11544 0 19 - 1132676 09:32:01 00:01:58 /usr/local/sbin/syslog-ng
20120209 053101 root 11545 11544 0 19 - 1159172 09:47:01 00:02:01 /usr/local/sbin/syslog-ng
20120209 054601 root 11545 11544 0 19 - 1185140 10:02:01 00:02:05 /usr/local/sbin/syslog-ng
20120209 060101 root 11545 11544 0 19 - 1212596 10:17:01 00:02:08 /usr/local/sbin/syslog-ng
20120209 061601 root 11545 11544 0 19 - 1244612 10:32:01 00:02:12 /usr/local/sbin/syslog-ng
20120209 063101 root 11545 11544 0 19 - 1278132 10:47:01 00:02:16 /usr/local/sbin/syslog-ng
20120209 064601 root 11545 11544 0 19 - 1312416 11:02:01 00:02:20 /usr/local/sbin/syslog-ng
20120209 070101 root 11545 11544 0 19 - 1343452 11:17:01 00:02:24 /usr/local/sbin/syslog-ng
20120209 071601 root 11545 11544 0 19 - 1377180 11:32:01 00:02:28 /usr/local/sbin/syslog-ng
20120209 073101 root 11545 11544 0 19 - 1408764 11:47:01 00:02:32 /usr/local/sbin/syslog-ng
20120209 074602 root 11545 11544 0 19 - 1438876 12:02:02 00:02:35 /usr/local/sbin/syslog-ng
20120209 080101 root 11545 11544 0 19 - 1475788 12:17:01 00:02:40 /usr/local/sbin/syslog-ng
20120209 081601 root 11545 11544 0 19 - 1511372 12:32:01 00:02:44 /usr/local/sbin/syslog-ng
20120209 083101 root 11545 11544 0 19 - 1554716 12:47:01 00:02:49 /usr/local/sbin/syslog-ng
So you can see that the memory footprint grows constantly.
Evan.
________________________________________
From: syslog-ng-bounces at lists.balabit.hu [syslog-ng-bounces at lists.balabit.hu] On Behalf Of Martin Holste [mcholste at gmail.com]
Sent: Wednesday, February 08, 2012 8:58 AM
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] 3.3.4 anyone else notice a memory leak
Hm, I'm not seeing the leak in ELSA (which uses program()) and I'm
using 3.3.4 from the latest tarball (not git).
On Tue, Feb 7, 2012 at 11:49 PM, Evan Rempel <erempel at uvic.ca> wrote:
> It looks like it must be writing to a program destination, or writing to a pipe that leaks the memory.
>
> The only syslog-ng processes I have that do not leak memory are;
>
> - read from a pipe and writes to files.
> - read from ksm /dev/log, write to files and tcp destination
>
> The rest leak;
>
> - Read from network, write to many pipes
> - Read from pipe and write to program
>
> Evan.
> ________________________________________
> From: syslog-ng-bounces at lists.balabit.hu [syslog-ng-bounces at lists.balabit.hu] On Behalf Of Evan Rempel [erempel at uvic.ca]
> Sent: Tuesday, February 07, 2012 9:35 PM
> To: Syslog-ng users' and developers' mailing list
> Subject: [syslog-ng] 3.3.4 anyone else notice a memory leak
>
> I run a number of syslog-ng processes, and I am noticing that the processes that read from
> a pipe, and write to a program destination all seem to have a memory leak in them.
>
> The rate of leak seems to be dependent on the volume of log lines (not surprising).
>
> Has anyone else noticed this.
>
> My config is;
>
>
>
>
> @version: 3.3
>
> #
> # UVic syslog-ng configuration
>
> options {
> log_fifo_size(100000);
> use_fqdn(yes);
> keep_hostname(yes);
> chain_hostnames(yes);
> time_reap(60);
> time_reopen(5);
> flush_lines(1000);
> flush_timeout(1000);
> };
>
> source local { pipe("/var/log/syslog.pipes/syslogstats" log_iw_size(90000) log_fetch_limit(500) flags(no-parse)); };
> source int { internal(); };
>
> destination localsyslog { tcp("localhost" port(1514) localip(localhost) log_fifo_size(50000) template("<$PRI>$S_DATE syslogstats@$HOST syslogstats: $MSGONLY\n") template_escape(no) ); };
>
> destination syslog_stats { program("/home/sysprog/syslog-apps/syslog_stats/syslogstats" template("$MESSAGE\n") template_escape(no) ); };
>
> log { source(local); destination(syslog_stats); flags(flow-control); };
> log { source(int); destination(localsyslog); };
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq
More information about the syslog-ng
mailing list