On Thu, 13 Oct 2011 15:32:27 +0200, Balazs Scheidler wrote:
Anything else looks interesting?
During this run, VmRSS kept rising, from: Sun Oct 9 21:44:51 CEST 2011; VmRSS: 3804 kB (...) to: Sun Oct 9 23:07:44 CEST 2011; VmRSS: 445332 kB
Valgrind itself allocates memory to store the errors and warnings. Do you see this increase of memory usage without valgrind too?
I did, that graph I sent here earlier was exactly the VmRSS of syslog-ng running, and as you've seen, it was rising. And it was running without valgrind :) I wanted to test this some more, but today I've noticed something odd. Since upgrading to 3.3.1, there is a strange behaviour of my system: if I reload syslog-ng, it spits out *tons* of similar messages: Internal error, duplicate configuration elements refer to the same persistent config; name='dd_queue(d_mesg,d_mesg#0)' As far as I can tell, this happens for every destination that was open: $ grep '^Oct 14 00:00:03' /var/log/messages | grep -c 'dd_queue(d_mesg,d_mesg#0)' 186 $ grep '^Oct 14 00:00:03' /var/log/messages | grep -c 'dd_queue(d_cron,d_cron#0)' 164 $ grep '^Oct 14 00:00:03' /var/log/messages | grep -c '_errorlog#0' 62435 I'm pretty sure there are no duplicate configuration elements in my config, especially for those "standard" ones like d_mesg or d_cron: $ grep d_mesg /etc/syslog-ng/syslog-ng.conf destination d_mesg { file("/var/log/messages" group("root") flush_lines(1)); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; $ grep messages /etc/syslog-ng/syslog-ng.conf destination d_mesg { file("/var/log/messages" group("root") flush_lines(1)); }; $ If there were, I'd get some errors at startup, right? Well, I don't. But what's most peculiar, is that after such reload, VmRSS stays roughly the same (at least it's not climbing like crazy).
I've also read the complete log and I've found two minor leaks, both of which leak memory when processing SIGHUPs, I've fixed these:
I've built a package with these two patches applied, and run it on my test system, to confirm this weird behaviour above. It still manifests. After a clean start, until there are logs to process, I can reload syslog-ng and there are no "Internal error" messages. As soon as I make my log relays send some messages to this test system, and try to reload it: boom, internal errors again. This is very weird, I don't recall this behaviour before. I know this description isn't the most precise one, but at this moment I had no time to investigate more. I promise I'll dig some more during next few days. But maybe even this much will help someone debug it. Regards, -- Jakub Jankowski|shasta@toxcorp.com|http://toxcorp.com/ GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D