[syslog-ng] Drops?

[Balazs Scheidler]

On Wed, 2009-01-14 at 09:39 -0600, Mike Rykowski wrote:
> I configured syslog-ng to pass messages to SEC via named pipes. When I
> did I saw in the log statistics:
> 
> Jan 13 09:10:00  syslog-ng[13736]: Log statistics; dropped='pipe(sec-
> pipe)=1607', dropped='unix-dgram(AF_UNIX(/dev/log))=0',
> processed='center(queued)=1794661', processed='center
> (received)=1718990',
> 
> I then called SEC using the 'program' option instead of named pipes and
> I got:
> 
> Jan 13 10:14:13 syslog-ng[14031]: Log statistics; dropped='program
> (/usr/local/bin/sec.pl -input="-" -conf=/usr/local/etc/my-sec.conf)=0',
> dropped='unix-dgram(AF_UNIX(/dev/log))=0', processed='center
> (queued)=2552863', processed='center(received)=2447130',

I don't see your exact configuration but the two numbers need not be
equal. The number of "queued" may count one message several times when
the same message is sent to multiple destinations. The number of
"received" messages is the total number of messages received from any of
the channels.

And 'dropped' messages are counted into 'queued' since messages are
dropped at the destination, 'queued' is counted before that.

> 
> The drops were zero but the queued and received were not equal. Does
> this mean messages are getting dropped?
> 
> When I don't call SEC at all I get queued and received to be equal:
> 
> Jan 14 09:26:44 syslog-ng[14119]: Log statistics; dropped='unix-dgram
> (AF_UNIX(/dev/log))=0', processed='center(queued)=70516726',
> processed='center(received)=70516726',

Your incoming logs may differ in properties, and if you are filtering on
those properties then these numbers may be different.

Also, I know that I keep repeating this these days, but as it seems the
problems nowadays are always at a corner of syslog-ng that got improved
in the shiny new 3.0 release. 3.0 got a new statistics infrastructure
which makes it easier to track down problems like this:

http://bazsi.blogs.balabit.com/2008/11/syslog-ng-statistics.html

-- 
Bazsi