[syslog-ng] performance

Jeffrey W. Baker jwbaker@acm.org
Mon, 08 May 2000 10:35:44 -0700 (PDT)

On Mon, 8 May 2000, Balazs Scheidler wrote:

> > > 	How syslog-ng deals with high-volume logging? The old syslogd is very bad
> > > in this respect; if you feed it with lots of logs, the system slows down
> > > horribly.
> > > 	So, how does syslog-ng behaves in such cases?
> > 
> > I find that syslog-ng drops approximately 95% of all log entries when the
> > system generates them quickly.  This is using the tcp protocol and writing
> > to a file on the remote machine.
> > 
> > This has kept me from trying to implement syslog-ng in production.
> You can finetune garbage collection parameters using the gc_idle_threshold
> and gc_busy_threshold options.
> idle threshold should be set to a low value, busy_threshold should be set to
> a high value (about the number of messages in a burst)
> options { gc_idle_threshold(100); gc_busy_threshold(5000); };

I tried your suggestion before.  I does help with transient message
bursts, but at steady state syslog-ng eventually gets overwhelmed.  My
application does not generate short, separated bursts of messages.  It
creates long, sustained streams of messages.  This is where I see
syslog-ng dropping (most) log entries.  You can try for yourself by
dumping short messages to an input named pipe in a tight loop.