On Mon, 8 May 2000, Balazs Scheidler wrote:
How syslog-ng deals with high-volume logging? The old syslogd is very bad in this respect; if you feed it with lots of logs, the system slows down horribly. So, how does syslog-ng behaves in such cases?
I find that syslog-ng drops approximately 95% of all log entries when the system generates them quickly. This is using the tcp protocol and writing to a file on the remote machine.
This has kept me from trying to implement syslog-ng in production.
You can finetune garbage collection parameters using the gc_idle_threshold and gc_busy_threshold options.
idle threshold should be set to a low value, busy_threshold should be set to a high value (about the number of messages in a burst)
options { gc_idle_threshold(100); gc_busy_threshold(5000); };
I tried your suggestion before. I does help with transient message bursts, but at steady state syslog-ng eventually gets overwhelmed. My application does not generate short, separated bursts of messages. It creates long, sustained streams of messages. This is where I see syslog-ng dropping (most) log entries. You can try for yourself by dumping short messages to an input named pipe in a tight loop. -jwb