[syslog-ng] dropping

[Nate Campi]

On Fri, Oct 07, 2005 at 08:51:27AM -0400, Mike wrote:
> On Fri, 7 Oct 2005, Ferenc Wagner wrote:
> 
> > Using Debian Sarge I set up a configuration where some 160
> > machines log by TCP to a single central server.  When the
> > machines boot (all at the same time) they obviously put
> > quite some load on the server, which results in lines like
> > 
> > Oct  6 20:55:18 bigyo syslog-ng[24969]: STATS: dropped 1303
> > 
> > after the client connected messages.  Also there is a
> > constant periodic loss (the clients run synchronised, so
> > cron jobs fire simultaneously) amounting to
>  
> I think the best thing to do is to stager the sending times of the 
> data...but failing that, adjust your system level buffer sizes.

Good idea, you should work hard to do this, it's probably the best thing
you can do to help if your problem is the bursts.
 
> this site talks a bit about doing that 
> http://www-didc.lbl.gov/TCP-tuning/linux.html, if you have never adjusted 
> the sizes before.
 
No TCP tuning is needed when the application is dropping the packets.
That STATS messages from syslog-ng mean that it's traversed the network
stack and the data is now in the hands of syslog-ng. The messages are
sent to destinations by syslog-ng and if/when the data from the
source(s) exceeds the ability of the destination(s) to accept them,
syslog-ng drops the message(s) and reports it in a STATS message.

To increase the performance writing to disk, increase the number of
lines buffered before writing using the sync() option and/or get faster
disks (maybe set up as RAID5 or even better RAID 0+1). Buffering too
much makes the risk of data loss due to crash a real risk, but a risk of
loss is acceptable if it allows you to not constantly lose data during
normal operation!

We also have a performance tips section of the FAQ:

 http://www.campin.net/syslog-ng/faq.html#perf

If you have a lot of regexps in your filters then the limit of messages
you can process under heavy load is decreased - see that URL.
-- 
Nate

"It is better to deserve honours and not have them than to have them and 
not deserve them." - Samuel Clemens