I think the best thing to do is to stager the sending times of the data...but failing that, adjust your system level buffer sizes. this site talks a bit about doing that http://www-didc.lbl.gov/TCP-tuning/linux.html, if you have never adjusted the sizes before. we ran into the same problem, and simply adjusting these values worked wonders (I think we set ours to be 256MByte max). Mike On Fri, 7 Oct 2005, Ferenc Wagner wrote:
Hi,
Using Debian Sarge I set up a configuration where some 160 machines log by TCP to a single central server. When the machines boot (all at the same time) they obviously put quite some load on the server, which results in lines like
Oct 6 20:55:18 bigyo syslog-ng[24969]: STATS: dropped 1303
after the client connected messages. Also there is a constant periodic loss (the clients run synchronised, so cron jobs fire simultaneously) amounting to
Oct 7 06:35:27 bigyo syslog-ng[24969]: STATS: dropped 9
Is there a way to overcome this? In average the log traffic is fairly low, but huge bursts do happen as described above. Setting log_fifo_size on the server didn't help much; it logs straight onto disk:
[stock Debian Sarge part distributing local logs elided] options { keep_hostname (yes); }; source s_cl { tcp (max_connections (255)); }; destination d_cl { file ("/var/log/cluster/$HOST" template ("$DATE $MSG\n") group ("adm") perm (0640) create_dirs (yes) dir_perm (750)); }; log { source (s_cl); destination (d_cl); };
The clients are configured like this (full file): options { use_dns (no); }; source s_all { internal (); unix-stream ("/dev/log"); file ("/proc/kmsg" log_prefix ("kernel: ")); }; destination bigyo { tcp ("bigyo"); }; log { source (s_all); destination (bigyo); };
Stock Sarge syslog-ng 1.6.5 with Debian patches on all machines. -- Thanks, Feri. _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html