[syslog-ng] performance

[Balazs Scheidler]

> > > > 	How syslog-ng deals with high-volume logging? The old syslogd is very bad
> > > > in this respect; if you feed it with lots of logs, the system slows down
> > > > horribly.
> > > > 	So, how does syslog-ng behaves in such cases?
> > > 
> > > I find that syslog-ng drops approximately 95% of all log entries when the
> > > system generates them quickly.  This is using the tcp protocol and writing
> > > to a file on the remote machine.
> > > 
> > > This has kept me from trying to implement syslog-ng in production.
> > 
> > You can finetune garbage collection parameters using the gc_idle_threshold
> > and gc_busy_threshold options.
> > 
> > idle threshold should be set to a low value, busy_threshold should be set to
> > a high value (about the number of messages in a burst)
> > 
> > options { gc_idle_threshold(100); gc_busy_threshold(5000); };
> 
> I tried your suggestion before.  I does help with transient message
> bursts, but at steady state syslog-ng eventually gets overwhelmed.  My
> application does not generate short, separated bursts of messages.  It
> creates long, sustained streams of messages.  This is where I see
> syslog-ng dropping (most) log entries.  You can try for yourself by
> dumping short messages to an input named pipe in a tight loop.

Then you may need to increase the output fifo size using the log_fifo_size()
option.



-- 
Bazsi
PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
     url: http://www.balabit.hu/pgpkey.txt