[syslog-ng] syslog-ng droping events

[Balazs Scheidler]

On Fri, 2006-11-10 at 01:11 -0800, Arya, Manish Kumar wrote:
> Hi,
> 
>    I have 3 syslog listeners (solaris 10 ) collecting
> log events from about 20,000 devices and fwding them
> on a central syslog box (solaris 10, 16 gb RAM and 3TB
> SAN storage) . the central syslog box stores them in
> oracle db and on filesystem.
> 
>    I notice that events from many devices are being
> droped on central syslog box.
>    I thought it might be because of udp protocol, but
> even after enabling tcp its droping events. I can see
> that events come on listeners boxes (I created temp
> file logs there)

Where does it drop messages? Some events are missing from the SQL
database, or some are missing from simple files too? 

Message dropping can happen for two reasons:
* messages are dropped from the input receive buffer because syslog-ng
is too slow to receive messages, _or_
* messages are actively dropped by syslog-ng after being successfully
received, because a specific target cannot cope with the load

You need to identify which scenario is happening and then take
appropriate measures. (increase socket buffer, increase log_fifo_size(),
increase DB performance, or enable flow_control and ensure the complete
log patch is flow-controlled)

-- 
Bazsi