[syslog-ng] Ongoing 3.3.4 memory leaks

Balazs Scheidler bazsi77 at gmail.com
Mon Apr 2 22:03:24 CEST 2012 

----- Original message -----
> Thanks for the quick reply.   See my comments below.
> 
> > -----Original Message-----
> > From: syslog-ng-bounces at lists.balabit.hu [mailto:syslog-ng-
> > bounces at lists.balabit.hu] On Behalf Of Balazs Scheidler
> > Sent: Saturday, March 31, 2012 2:22 PM
> > To: Syslog-ng users' and developers' mailing list
> > Subject: Re: [syslog-ng] Ongoing 3.3.4 memory leaks
> > 
> > On Fri, 2012-03-30 at 16:52 -0500, Marvin Nipper wrote:
> > > Hi.
> > > 
> > > Sorry, looking for some help.   Still trying to chase some memory
> > leaks
> > > that I see on Solaris 10.   I’ll crank up syslog-ng, and it’ll start
> > > running at around 7 or 8MB, and interestingly, it will run that way
> > > sometimes for several hours, and then (for reasons unknown), I’ll
> > take
> > > a look at it a bit later, and suddenly it will have jumped to 26MB
> > (or
> > > later on, maybe 75MB).   I’ve already applied the (older) logwriter.c
> > > fix and the (new) Gergely driver.h/afsocket.c fix, in an attempt to
> > > resolve known issues.   Certainly the logwriter fix corrected a
> > > super-nasty, continuous leak problem, but this other leak problem
> > > still lingers.
> > 
> > I'm not sure which fixes you've applied. Can you post a better
> > reference?
> [Marvin Nipper] 
> They are the logwriter fix that you mentioned just below, and also the
> afsocket patch that you were discussing this weekend in the "Memory leak
> reintroduced in 3.3.3" thread.   However, based upon your comments (in
> that thread), I plan to "unwind" that set of changes, as it seems like
> those may not have been legit.
> 
> 

I've processed those after answering your email.

> 
> > 
> > I've applied only one patch after v3.3.4 got released which fixes a
> > memory leak, this one:
> > 
> > logwriter: fixed a memory leak for messages that couldn't be
> > flushed at once
> > 
> > Sometimes a message is dequeued, and then can't be posted to the
> > destination
> > (probably because of an EAGAIN or something like that). In this
> > case
> > it is readded back to the queue. In this case a reference was lost,
> > causing
> > a memory leak, but not for all messages, but only for the affected
> > ones.
> > 
> > Reported-By: Evan Rempel <erempel at uvic.ca>
> > Signed-off-by: Balazs Scheidler <bazsi at balabit.hu>
> > 
> > I can't remember anything else, hopefully I haven't missed patches by
> > Gergely.
> > > 
> > > MY PROBLEM is that I’m wanting to use the debug symbols in syslog-ng,
> > > so that I can have the Solaris tools specifically show the exact
> > > location where these leaks are happening, BUT, whereas a non-
> > debugging
> > > version of syslog-ng executes properly, when I configure syslog-ng
> > > with –enable-debug, the resulting executable will no longer start.
> > It
> > > immediately exits, coughing up what amounts to a syntax error (even
> > > though I’m using the exact same command-line invocation that I use on
> > > the non-debugged executable):
> > > 
> > > 
> > > #/etc/init.d/syslog-ng start
> > > syslog-ng service starting.
> > > mkfifo: File exists
> > > mkfifo: File exists
> > > Usage: /usr/local/sbin/syslog-ng { start | stop }
> > > #
> > > 
> > > Something about enabling debugging is impacting the operational
> > > behavior of the resulting binary.
> > 
> > Hmm.   That output is strange, you said you were running Solaris, but
> > Solaris uses SMF for service management. Yet, you executed an sysv init
> > script, which then complained that /usr/local/sbin/syslog-ng has
> > different options, even though /usr/local/sbin/syslog-ng is probably
> > your syslog-ng executable (or at least that's the default path).
> > 
> > Can you perhaps start syslog-ng by hand?
> > 
> > /usr/local/sbin/syslog-ng -F
> > 
> > should do it.
> [Marvin Nipper] 
> Actually, I had already done that, but here's that output, just to show
> you: # /usr/local/sbin/syslog-ng \
> > -F -f /etc/syslog-ng/syslog-ng.conf \
> > -p /var/run/syslog-ng.pid \
> > --persist-file=/usr/local/syslog-ng/syslog-ng.persist
> Usage: /usr/local/sbin/syslog-ng { start | stop }
> 

this isn't syslog-ng that produces this output. maybe the installation overwrote it? is that file a shell script? 

> 
> 
> 
>   Debug symbols shouldn't affect how things run.
> > --enable-debug only adds a couple of options to CFLAGS, and only if the
> > compiler is gcc.
> > 
> > if test "x$ac_compiler_gnu" = "xyes"; then
> > CFLAGS="${CFLAGS} -Wall"
> > if test "x$enable_debug" = "xyes"; then
> > CFLAGS="${ac_cv_env_CFLAGS_value} -Wall -g"
> > fi
> > 
> > if test "x$enable_gprof" = "xyes"; then
> > CFLAGS="${CFLAGS} -pg"
> > fi
> > if test "x$enable_gcov" = "xyes"; then
> > CFLAGS="${CFLAGS} -fprofile-arcs -ftest-coverage"
> > fi
> > fi
> > CFLAGS="${CFLAGS} -pthread"
> > 
> > If nothing else, try passing -g in CFLAGS before running configure,
> > without
> > passing --enable-debug.
> [Marvin Nipper] 
> I actually didn't even know that the --enable-debug option existed in
> the syslog-ng config, and so I started this effort by doing a
> compilation with -g in CFLAGS, and that's when I first encountered the
> syntax issue (that results in the "Usage" error).   When that goofy
> result occurred, I figured that I'd done something wrong, and then
> stumbled into the debug capability "built into" your configure
> scripting, so I tried that next, which ends up with the same result.
> 
> So, I don't know if this is yet another "goofy Solaris problem", but
> some aspect of enabling the debug symbols, seems to frak up some aspect
> of processing the command line at startup.
> 
> If you have any other ideas or thoughts, I'll certainly take them.   I
> appreciate your time and input.
> 
> 
> 
> > 
> > > 
> > > Anyway, I can use (for Solaris) libumem and mdb to narrow this down
> > to
> > > a particular module, but without the debug symbols, I can’t really
> > get
> > > to a specific chunk of code, so it would obviously help to get that
> > > working.   If anyone has any ideas, I certainly would appreciate the
> > > input.
> > > 
> > 
> > It'd really be appreciated if you could point out where the leak is
> > happening. It might be use-case specific or some failure handler or
> > whatever. Let's hope that starting syslog-ng from the command line
> > works.
> > 
> > --
> > Bazsi
> > 
> > 
> > _______________________________________________________________________
> > _______
> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Documentation:
> > http://www.balabit.com/support/documentation/?product=syslog-ng
> > FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> 
> 
> 
> This e-mail may contain confidential and/or privileged information. If
> you are not the intended recipient (or have received this e-mail in
> error) please   notify the sender immediately and destroy this e-mail.
> Any unauthorized   copying, disclosure or distribution of the material in
> this e-mail is strictly   forbidden.
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng FAQ:
> http://www.balabit.com/wiki/syslog-ng-faq
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20120402/5dbfffb7/attachment-0001.htm

More information about the syslog-ng mailing list