On Fri, Oct 21, 2005 at 06:54:04AM -0700, Scott C wrote:
But what's really most peculiar in this scenario is the fact that the numbers simply don't add up. Why does syslog-ng appear (on the surface) to be dropping a very large percentage of the messages that it receives? I realize that it's not, but the numbers tell a different story. And how could it possibly drop so many messages when the FIFO queue is configured to buffer three million lines? How preposterous!
So you think you really have all the logs but you see STATS messages reporting dropped messages? How would you know if you really have them all? It's possible that under heavy load you have some program or pipe destination (or maybe even file if you have slow disks) that just can't keep up. That's not syslog-ng's fault, it just lets you know that the buffer filled up. Right now all anyone can do is shoot off wild guesses like mine above, since there's no hard data in your post, just your conclusions. If you want to post your syslog-ng.conf, output of system commands like "netstat -i", prstat, "iostat -mnPxz 10" and vmstat during peak loads, and whatever else you used to reach your conclusions then we'd be in a better position to help. OBTW there are performance tips in the FAQ that give clues as to causes: http://www.campin.net/syslog-ng/faq.html#perf Possible culprits: DNS, regexps (though you say CPU is ok, so maybe not), logging to a tty or the console. -- Nate "I must've seen it in a USENET posting; that's sort of like hearsay evidence from Richard Nixon..." - Houghton, Blair