[syslog-ng] 1.6.5 performance

[Nate Campi]

On Fri, Oct 21, 2005 at 06:54:04AM -0700, Scott C wrote:
> But what's really most peculiar in this scenario is the fact that the
> numbers simply don't add up.  Why does syslog-ng appear (on the
> surface) to be dropping a very large percentage of the messages that
> it receives?  I realize that it's not, but the numbers tell a
> different story.  And how could it possibly drop so many messages when
> the FIFO queue is configured to buffer three million lines?  How
> preposterous!
  
So you think you really have all the logs but you see STATS messages
reporting dropped messages? How would you know if you really have them
all? It's possible that under heavy load you have some program or pipe
destination (or maybe even file if you have slow disks) that just can't
keep up. That's not syslog-ng's fault, it just lets you know that the
buffer filled up.

Right now all anyone can do is shoot off wild guesses like mine above,
since there's no hard data in your post, just your conclusions. If you
want to post your syslog-ng.conf, output of system commands like
"netstat -i", prstat, "iostat -mnPxz 10" and vmstat during peak loads,
and whatever else you used to reach your conclusions then we'd be in a
better position to help.

OBTW there are performance tips in the FAQ that give clues as to causes:

 http://www.campin.net/syslog-ng/faq.html#perf

Possible culprits: DNS, regexps (though you say CPU is ok, so maybe
not), logging to a tty or the console.
-- 
Nate

"I must've seen it in a USENET posting; that's sort of like hearsay
evidence from Richard Nixon..." - Houghton, Blair