On Wed, 2009-01-14 at 09:39 -0600, Mike Rykowski wrote:
I configured syslog-ng to pass messages to SEC via named pipes. When I did I saw in the log statistics:
Jan 13 09:10:00 syslog-ng[13736]: Log statistics; dropped='pipe(sec- pipe)=1607', dropped='unix-dgram(AF_UNIX(/dev/log))=0', processed='center(queued)=1794661', processed='center (received)=1718990',
I then called SEC using the 'program' option instead of named pipes and I got:
Jan 13 10:14:13 syslog-ng[14031]: Log statistics; dropped='program (/usr/local/bin/sec.pl -input="-" -conf=/usr/local/etc/my-sec.conf)=0', dropped='unix-dgram(AF_UNIX(/dev/log))=0', processed='center (queued)=2552863', processed='center(received)=2447130',
I don't see your exact configuration but the two numbers need not be equal. The number of "queued" may count one message several times when the same message is sent to multiple destinations. The number of "received" messages is the total number of messages received from any of the channels. And 'dropped' messages are counted into 'queued' since messages are dropped at the destination, 'queued' is counted before that.
The drops were zero but the queued and received were not equal. Does this mean messages are getting dropped?
When I don't call SEC at all I get queued and received to be equal:
Jan 14 09:26:44 syslog-ng[14119]: Log statistics; dropped='unix-dgram (AF_UNIX(/dev/log))=0', processed='center(queued)=70516726', processed='center(received)=70516726',
Your incoming logs may differ in properties, and if you are filtering on those properties then these numbers may be different. Also, I know that I keep repeating this these days, but as it seems the problems nowadays are always at a corner of syslog-ng that got improved in the shiny new 3.0 release. 3.0 got a new statistics infrastructure which makes it easier to track down problems like this: http://bazsi.blogs.balabit.com/2008/11/syslog-ng-statistics.html -- Bazsi