[syslog-ng] syslog-ng 2.0.5 dropped messages - totally confused

[Geller, Sandor (IT)]

Hi,

> Anyhow. On the two central servers, I different numbers of 
> records in the files, and the statistics
> on the sender show
> 
> 2008-03-04T23:26:38-08:00 local at caribou.comp.uvic.ca 
> syslog.info syslog-ng[3391]: Log statistics; 
> dropped='tcp(AF_INET(server2:514))=0',
> dropped='tcp(AF_INET(server1:514))=14690649',
> processed='center(queued)=53993217',
> processed='center(received)=17997739',
> processed='destination(syslogServer2)=17997739',
> processed='destination(syslogServer1)=17997739',
> processed='destination(syslog)=17997739',
> processed='source(local)=17997739'

Please note that these numbers are aggregated, so these show the
amount of messages arriving to/ leaving syslog-ng since it has been
started.

> The problem is that the files on disk show
> 
> caribou	16257954
> server1	1742054
> server2	965475
> 
> 
> and that just doesn't add up. Neither server shows any 
> dropped messages.

I think you misunderstood what dropped means. Dropped stands for
messages which were undeliverable (maybe the TCP connection was
broken you can check the internal messages of syslog-ng for such
problems) so dropped messages never reached server1 or server2.

> I know that the statistics miss some time at the beginning of the day
> and at the end of the day, but the numbers don't even come close.

The statistics shows everything which reached syslog-ng.

> Caribou stats show that no messages are dropped to server2, 
> and lots dropped
> to server1, however, server2 actually wrote more messages to 
> disk. Server 1
> is also about 30% faster than server2.

As I wrote above when the report shows that a destination had more
dropped messages then that server received fewer messages than the
other.

Using the numbers above: destination(syslogServer1) processed
17997739 while the TCP destination dropped 14690649 messages.
Although the TCP destination could be used by multiple destinations
I guess destination(syslogServer1) and TCP(syslog1) could be mapped.
This means that 3307090 messages were delivered to syslogServer1.
And as this is still an aggregated number you should calculate the
number of messages starting from the time when the central log server
has been started so there is a chance that you have to check multiple
logfiles on your servers because the period you're interested in may
span multiple days.

Regards,

Sandor
--------------------------------------------------------

NOTICE: If received in error, please destroy and notify sender. Sender does not intend to waive confidentiality or privilege. Use of this email is prohibited when received in error.