On Fri, Oct 07, 2005 at 08:51:27AM -0400, Mike wrote:
On Fri, 7 Oct 2005, Ferenc Wagner wrote:
Using Debian Sarge I set up a configuration where some 160 machines log by TCP to a single central server. When the machines boot (all at the same time) they obviously put quite some load on the server, which results in lines like
Oct 6 20:55:18 bigyo syslog-ng[24969]: STATS: dropped 1303
after the client connected messages. Also there is a constant periodic loss (the clients run synchronised, so cron jobs fire simultaneously) amounting to
I think the best thing to do is to stager the sending times of the data...but failing that, adjust your system level buffer sizes.
Good idea, you should work hard to do this, it's probably the best thing you can do to help if your problem is the bursts.
this site talks a bit about doing that http://www-didc.lbl.gov/TCP-tuning/linux.html, if you have never adjusted the sizes before.
No TCP tuning is needed when the application is dropping the packets. That STATS messages from syslog-ng mean that it's traversed the network stack and the data is now in the hands of syslog-ng. The messages are sent to destinations by syslog-ng and if/when the data from the source(s) exceeds the ability of the destination(s) to accept them, syslog-ng drops the message(s) and reports it in a STATS message. To increase the performance writing to disk, increase the number of lines buffered before writing using the sync() option and/or get faster disks (maybe set up as RAID5 or even better RAID 0+1). Buffering too much makes the risk of data loss due to crash a real risk, but a risk of loss is acceptable if it allows you to not constantly lose data during normal operation! We also have a performance tips section of the FAQ: http://www.campin.net/syslog-ng/faq.html#perf If you have a lot of regexps in your filters then the limit of messages you can process under heavy load is decreased - see that URL. -- Nate "It is better to deserve honours and not have them than to have them and not deserve them." - Samuel Clemens