On Sat, 12 Feb 2005 13:04:32 +0100, Michael Arndt <M.Arndt@science-computing.de> wrote:
thx for this interesting quantitaive Info.
On a related note, I believe that the reason syslog-ng handles this volume with little or no loss is related directly to how the daemon queues up packets and writes the data to disk in blocks instead of making a single write() call per message. I'll post the relevant sections of my syslog-ng.conf if there is interest.
Two additional Questions:
what are your network specs : bandwith client -> loghost ?
All of the sources are in the same physical facility, connected to the loghost via a dedicated 100/Full interface to a switch which only serves the loghost. Currently this is a 2924 switch, soon to be replaced with a 3524. Even at peak moments, the actual bandwidth seldom exceeds 5 megabits -- the real issue seems to be PPS.
and are you seeing traces of dropped messages on the clients or the server ?
I've done some primitive load testing with sending UDP syslog packets each containing a monotonically increasing sequence number, and the limiting factor when using UDP seems to be related to packets-per-second rather than bandwidth. From 'netstat -s', I see about 200,000 packets/year logged as "dropped due to full socket buffers". I've tuned the value of net.inet.udp.recvspace as high as it can safely be set. From the overall volume, this works out to about %0.001 loss. Kevin Kadow