[syslog-ng] Possible memleak in 3.3 HEAD

[Balazs Scheidler]

Hi,

Thanks for the detailed problem description Jakub and the diagnosis
Gergely. I've committed this patch to master:

commit 2ed7f92153aec5e4e666ea17dcd8d2232a8e76f9
Author: Balazs Scheidler <bazsi at balabit.hu>
Date:   Tue Sep 27 20:12:24 2011 +0200

    log_dest_driver_release_queue(): unref the queue even if no name is present
    
    This patch fixes a memory leak for file destinations, in which we leak all
    LogQueue instances, possibly containing data, which can become quite large,
    especially if a lot of log files are opened/closed continously.
    
    Reported-By: Jakub Jankowski <shasta at toxcorp.com>
    Cc: <syslog-ng-stable at balabit.hu>
    Cc: Gergely Nagy <algernon at balabit.hu>
    Signed-off-by: Balazs Scheidler <bazsi at balabit.hu>

While I was there, I've also fixed a related problem, although less likely to happen:

commit e224d45da2ecad68f7f9c44895849a0fc795aae5
Author: Balazs Scheidler <bazsi at balabit.hu>
Date:   Tue Sep 27 20:14:35 2011 +0200

    log_dest_driver_acquire_queue(): make sure all dest drivers have a name
    
    The persist_name for acquire_queue() identifies the destination driver that
    would like to keep its queue kept accross reloads.
    
    File destinations didn't keep their queue, effectively causing file drivers
    to lose their queue accross reloads. This went unnoticed as LogWriter does
    one last time to flush everything out to the disk when reloading, but
    still it is more important to keep the queue properly.
    
    Cc: syslog-ng-stable at balabit.hu
    Signed-off-by: Balazs Scheidler <bazsi at balabit.hu>


On Mon, 2011-09-19 at 14:16 +0200, Gergely Nagy wrote:
> Jakub Jankowski <shasta at toxcorp.com> writes:
> 
> > It looks like 3.2.2 does not exhibit the leak I'm seeing on 3.3. Complete 
> > valgrind logs are available here: http://toxcorp.com/stuff/syslog-ng-leak/
> >
> > Let me know if I can provide anything else to help with this. Thanks!
> 
> Quick update: I managed to reproduce the problem, and I can verify that
> this is present in current git head indeed.
> 
> I used the following config:
> 
> ,----
> | @version: 3.3
> | @include "scl.conf"
> | 
> | options {
> | 	time-reap(1);
> | 	create-dirs(yes);
> | 	frac-digits(4);
> | };
> | source s_network {
> | 	tcp(port (13514) tags("tcp-tag") flags(no-parse));
> | };
> | destination d_file {
> | 	file("/tmp/many-file/${FIRST:-default}-seq-${UNIXTIME}.log"
> | 		template("${REST}\n")
> | 	);
> | };
> | log {
> | 	source(s_network);
> | 	destination(d_file);
> | };
> `----
> 
> Then do the following: for f in $(seq 1 10000); do echo "host-$f message $f" | nc localhost 13514; done
> 
> I have an idea where to look further, will see how far I get.
> 

-- 
Bazsi