[syslog-ng] UDP errors and lost UDP messages

[Matt Zagrabelny]

Greetings list!

Using syslog-ng 3.1 with Debian Squeeze, 2.6.32-5-amd64. The system
has 8GB of RAM.

I'm losing some UDP logs. I know to not use UDP - we use TLS for our
Debian systems, but our Cisco gear leaves us with few options.

According to netstat, the rate is anywhere from 600 to 3000 UDP errors
per second. Using a tcpdump query of "dst port 514" show about the
same rate of UDP traffic coming to the system.

I've bumped the buffer size according to various docs:
$ head -n -0 /proc/sys/net/core/[rw]mem_*
==> /proc/sys/net/core/rmem_default <==
16777216

==> /proc/sys/net/core/rmem_max <==
16777216

==> /proc/sys/net/core/wmem_default <==
16777216

==> /proc/sys/net/core/wmem_max <==
16777216

And the udp specific memory limits:

$ head -n -0 /proc/sys/net/ipv4/*udp*
==> /proc/sys/net/ipv4/udp_mem <==
768384 1024512 1536768

==> /proc/sys/net/ipv4/udp_rmem_min <==
16777216

==> /proc/sys/net/ipv4/udp_wmem_min <==
16777216

My UDP source for syslog-ng is also using a larger buffer:

$ grep -A4 -B1 'udp(' /etc/syslog-ng/syslog-ng.conf
source s_udp {
    udp(
        keep_hostname(yes)
        so_rcvbuf(16777216)
    );
};

According to syslog-ng-ctl stats the system is processing ~270 UDP
messages per second. This hasn't really changed since I've made the
kernel variable tweaks, nor after changing the so_rcvbuf size either.

Any ideas of what to look for next?

Thanks!

-m