[syslog-ng] High throughput UDP logging configuration.

[László Várady (lvarady)]

Hi,

> 1. The OS UDP buffer seems to be 128MB in size and the so_rcvbuf configured ins 64M in size. Is that because the syslog-ng configuration of so_rcvbuf is in characters but the OS buffer is in bytes?

This is because the kernel doubles the value set by syslog-ng (to allow space for bookkeeping overhead), and this doubled value is returned by getsockopt(2) and other tools.

> 3. Increasing the log_iw_size or the log_iw_size actually seems to make things worse.

These 2 values already seem high enough.
Disabling flow-control is also a good idea IMO, when using UDP sources.

> All suggestions that help me understand this and help to minimize the drops are welcome.

Could you share how incoming packets are distributed across the 8 sockets?

The default SO_REUSEPORT mechanism distributes packets based on the hash of (peer IP address, port) and (local IP address, port),
Hashing collision is also likely to happen [1], so if you encounter this problem, there are other possible resolutions. The commercial syslog-ng version has, for example, an
udp-balancer() driver, that uses custom BPF programs to achieve an even distribution of packets.


[1] https://blog.cloudflare.com/how-to-receive-a-million-packets/

--
László Várady
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20200528/d1c6a6cf/attachment.html>