[syslog-ng] Syslog-ng dropped packets.

[Balazs Scheidler]

On Fri, 2006-07-14 at 06:21 -0700, Vaibhav Goel wrote:
> Hi Bazsi
> 
> We are still having issues with dropped packets under heavy load.  We
> increased the size of rmem_default and rmem_max drastically but we are
> not seeing much of an improvement....Maybe about 50% or so.  Currently,
> the sizes are 
> 
> $ > cat /proc/sys/net/core/rmem_default
> 786426
> $ > cat /proc/sys/net/core/rmem_max
> 1572852
> 
> Do you have any other suggestions?  We are running syslog-ng 1.6.11.
> 
> All the logging is done over TCP (except 1 UDP source but that doesn't
> generate a lot of logging data).

It would be important to know whether it is syslog-ng itself that drops
messages or something else in the system.

Do you have STATS lines in your logs? That shows how many lines were
dropped by syslog-ng itself. It is also possible that it is the sending
syslog-ng process that drops messages (in the case of TCP), you can
verify this by checking the STATS line of the sending syslog-ng process.
You can improve this situation somewhat by increasing log_fifo_size on
the sender.

1.6.x sends messages on its TCP destination without flow control, which
means that if the bandwidth of a TCP channel is less than the messages
received, it could intentionally drop messages, that's what is shown in
the DROPPED line.

2.0 supports flow control, which means that syslog-ng is trying to slow
its inputs down to the rate of its output pipes. This could slow down
applications at the end, but this is the only way to avoid dropping
messages.



-- 
Bazsi