On Mon, 14 Feb 2005 08:37:20 +0100, Michael Arndt wrote:
Kevin Kadow wrote:
Michael Arndt wrote: and are you seeing traces of dropped messages on the clients or the server ?
From 'netstat -s', I . . . "dropped due to full socket buffers".
I've set syslog-ng to log stats hourly, it always reports "dropped 0", the clients are configured to just toss out UDP messages directly to the MAC address of the log (static ARP entry on the clients), and so are unaware of success or failure. The deployment was designed a few years ago to handle anticipated log volume and with a key design criteria that logging cannot impact the performance of the log sources; even at the cost of dropped events. For example, if a PIX is configured to use TCP instead of UDP, the log stream becomes reliable, but the PIX will stop passing traffic if the logger fails (See http://tinyurl.com/4xcve). This may be considered a feature in certain deployments.
On a related note, I believe that the reason syslog-ng handles this volume with little or no loss is related directly to how the daemon queues up packets and writes the data to disk in blocks instead of making a single write() call per message. I'll post the relevant sections of my syslog-ng.conf if there is interest.
i am definitely interested ;-)
Here's the relevant lines from my "options" block: options { stats(3600); sync(256); log_fifo_size(4095); log_msg_size(2048); }; Depending on the block sized used by your filesystem and the stripe size used by your RAID (if you are using hardware RAID), you might want to set these values significantly higher. My log messages are generally quite short, so there is a big performance gain from buffering many messages per write. If you are trying to debug a problem by watching logs "in realtime" (e.g. via "tail -f") the buffering done by these syslog-ng options may become a liability, as the logger may buffer as many as 256 lines before writing to disk. Kevin Kadow