[syslog-ng] performance

[Jeffrey W. Baker]

On Mon, 8 May 2000, Balazs Scheidler wrote:

> > > 	How syslog-ng deals with high-volume logging? The old syslogd is very bad
> > > in this respect; if you feed it with lots of logs, the system slows down
> > > horribly.
> > > 	So, how does syslog-ng behaves in such cases?
> > 
> > I find that syslog-ng drops approximately 95% of all log entries when the
> > system generates them quickly.  This is using the tcp protocol and writing
> > to a file on the remote machine.
> > 
> > This has kept me from trying to implement syslog-ng in production.
> 
> You can finetune garbage collection parameters using the gc_idle_threshold
> and gc_busy_threshold options.
> 
> idle threshold should be set to a low value, busy_threshold should be set to
> a high value (about the number of messages in a burst)
> 
> options { gc_idle_threshold(100); gc_busy_threshold(5000); };

I tried your suggestion before.  I does help with transient message
bursts, but at steady state syslog-ng eventually gets overwhelmed.  My
application does not generate short, separated bursts of messages.  It
creates long, sustained streams of messages.  This is where I see
syslog-ng dropping (most) log entries.  You can try for yourself by
dumping short messages to an input named pipe in a tight loop.

-jwb