[syslog-ng] syslog-ng 2.0.5 dropped messages - totally confused

[Evan Rempel]

Geller, Sandor (IT) wrote:
> Hi,
> 
>> Anyhow. On the two central servers, I different numbers of 
>> records in the files, and the statistics
>> on the sender show
>>
>> 2008-03-04T23:26:38-08:00 local at caribou.comp.uvic.ca 
>> syslog.info syslog-ng[3391]: Log statistics; 
>> dropped='tcp(AF_INET(server2:514))=0',
>> dropped='tcp(AF_INET(server1:514))=14690649',
>> processed='center(queued)=53993217',
>> processed='center(received)=17997739',
>> processed='destination(syslogServer2)=17997739',
>> processed='destination(syslogServer1)=17997739',
>> processed='destination(syslog)=17997739',
>> processed='source(local)=17997739'
> 
> Please note that these numbers are aggregated, so these show the
> amount of messages arriving to/ leaving syslog-ng since it has been
> started.

Yes, I have taken the first statistic in the day and subtracted if from the
last statistic in the day. There is some time between 00:00 and the first statistic,
and the last statistic and 23:59.999... This time will total the interval between
statistics. This is what I meant by "the statistics miss some time in the day."


>> The problem is that the files on disk show
>>
>> caribou	16257954
>> server1	1742054
>> server2	965475
>>
>>
>> and that just doesn't add up. Neither server shows any 
>> dropped messages.
> 
> I think you misunderstood what dropped means. Dropped stands for
> messages which were undeliverable (maybe the TCP connection was
> broken you can check the internal messages of syslog-ng for such
> problems) so dropped messages never reached server1 or server2.

I didn't misunderstand this. The source server stats show 17997739
messages queued, 14690649 dropped to the syslogServer1 which means
that 3307090 were delivered.

SyslogServer1 stats report that no messages were dropped, but countinmg
of log lines in the files on SyslogServer1 from caribou only
totals 1742054.

What happened to the other 1565036.

Even worse, the syslog-ng stats on caribou report 0 drops to syslogServer2
destination, but it only recored 965475 message in its files. Where did
the other 15292479 messages go?

I can't loose 15292479 without having any reporting on it.

> 
>> Caribou stats show that no messages are dropped to server2, 
>> and lots dropped
>> to server1, however, server2 actually wrote more messages to 
>> disk. Server 1
>> is also about 30% faster than server2.
> 
> As I wrote above when the report shows that a destination had more
> dropped messages then that server received fewer messages than the
> other.

I agree, so where did all of the messages for syslogServer2 go?


> Using the numbers above: destination(syslogServer1) processed
> 17997739 while the TCP destination dropped 14690649 messages.
> Although the TCP destination could be used by multiple destinations
> I guess destination(syslogServer1) and TCP(syslog1) could be mapped.
> This means that 3307090 messages were delivered to syslogServer1.
> And as this is still an aggregated number you should calculate the
> number of messages starting from the time when the central log server
> has been started so there is a chance that you have to check multiple
> logfiles on your servers because the period you're interested in may
> span multiple days.

I take this into account by taking the difference between the first and
last statistic record in each day. It misses a little, but not much.


Evan.