syslog-ng 2.0rc1 memory usage
Anyone running 2.0rc1 on Solaris 9 (sparc)? If so, what memory requirements are you seeing for the syslog-ng process? My install handles ~ 700 messages/sec (all UDP) and over the course of a day syslog-ng will gradually consume nearly all available free memory (I've seen it up to 1GB). (syslog-ng is bounced by logrotate daily, so the process repeats itself) I was able to substantially decrease the rate of consumption by adding the 'flush_lines" global (syslog-ng process now at 91MB and growing ... slowly). I installed 2.0rc1 on one server, kept another at 1.6.8 (roughly 500 messages/sec load). The syslog-ng process there is stable at 3 MB. Here's the config from the 2.0rc1 server: options { log_fifo_size(8192); flush_lines(4096); long_hostnames(off); use_dns(yes); use_fqdn(no); create_dirs(no); }; source network { udp(); }; filter f_messages { level(info..warn) and not facility(auth); }; destination messages { file("/var/log/messages"); }; log { source(network); filter(f_messages); destination(messages); }; syslog-ng 1.6.8 config identical except for options section: options { use_fqdn(no); keep_hostname(no); use_dns(yes); long_hostnames(off); sync(0); log_fifo_size(1000); }; The information transmitted (including attachments) is covered by the Electronic Communications Privacy Act, 18 U.S.C. 2510-2521, is intended only for the person(s) or entity/entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient(s) is prohibited. If you received this in error, please contact the sender and delete the material from any computer.
On Thu, 2006-07-20 at 19:47 -0500, Martin, David M wrote:
Anyone running 2.0rc1 on Solaris 9 (sparc)? If so, what memory requirements are you seeing for the syslog-ng process?
My install handles ~ 700 messages/sec (all UDP) and over the course of a day syslog-ng will gradually consume nearly all available free memory (I've seen it up to 1GB).
(syslog-ng is bounced by logrotate daily, so the process repeats itself) I was able to substantially decrease the rate of consumption by adding the 'flush_lines" global (syslog-ng process now at 91MB and growing ... slowly).
This definitely seems to be some kind of memory leak. Can you post a configuration file that reproduces the problem? Hopefully it is a leak in syslog-ng itself and not in one of the system libraries. -- Bazsi
On Thu, 2006-07-20 at 19:47 -0500, Martin, David M wrote:
Anyone running 2.0rc1 on Solaris 9 (sparc)? If so, what memory requirements are you seeing for the syslog-ng process?
My install handles ~ 700 messages/sec (all UDP) and over the course of a day syslog-ng will gradually consume nearly all available free memory (I've seen it up to 1GB).
(syslog-ng is bounced by logrotate daily, so the process repeats itself) I was able to substantially decrease the rate of consumption by adding the 'flush_lines" global (syslog-ng process now at 91MB and growing ... slowly).
I installed 2.0rc1 on one server, kept another at 1.6.8 (roughly 500 messages/sec load). The syslog-ng process there is stable at 3 MB.
I could not reproduce the leak so far locally, although I fixed some one-off allocation leaks but that should only affect reloads and not simply running the process. I suspect this is strict production environment, but I ask anyway, can you run something like valgrind on the syslog-ng process? Another idea is to use the built-in leak finder in syslog-ng, it is not as intrusive but might still help. If neither of these is possible, I could still use a core file dumped when syslog-ng is large in size. (I'd need debugging symbols on the syslog-ng executable as well) There must be some kind of usage pattern that I miss here, but without that I cannot fix the problem. -- Bazsi
participants (2)
-
Balazs Scheidler
-
Martin, David M