[syslog-ng] udp drops

[Jan Schaumann]

Balazs Scheidler <bazsi at balabit.hu> wrote:
 
> Hmm.. one possible problem is that syslog-ng wakes up too often
> processes a small number of messages and goes back to sleep. Since the
> poll iteration has its overhead, this might add up to be significant.
> 
> You could perhaps play with time_sleep(), I'd go for 30msecs which would
> limit syslog-ng to wake up at most 30 times per second.

That actually makes things a lot worse, as the buffers immediately fill
up and aren't drained quickly enough.

> Then, make sure that you actually have a large enough UDP receive
> buffer. so_rcvbuf() might not be enough, as systems usually add further
> limits on the maximum per-socket receive buffer size.

Yeah, that helps a lot.  I had initially resisted making those changes
as I was trying to see how/if I can tune syslog-ng to get the same
performance as regular syslog without any outside changes.
 
> fetch_limit() might be related, if you have only a small number of
> sources, you could increase that, but don't forget to adjust the
> destination window size, as described in the documentation:

That helps, too.

> syslog-ng core can do about 130k msg/sec without writing things to
> files, and about 70k/sec if you have a single destination file. however
> it might have a latency that causes the udp() receive buffer to fill up.
> If you carefully size your udp() receive buffer you can probably achieve
> no message losses for about 15k msg/sec.

With the above changes (and the fix for Bug #49, thank you very much),
I'm now getting syslog-ng down to between 2% and 4% UDP drops, which is
about the same as stock syslog was.

Well, it's slightly worse, since syslog-ng is now dropping
(intentionally) a large number of messages that stock syslog is
dutyfully writing to disk.  Also, the increase buffersize of course also
make stock syslog be more performant, but for now the above should be
acceptable until I have load balancing added.

Thanks!
-Jan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 186 bytes
Desc: not available
Url : http://lists.balabit.hu/pipermail/syslog-ng/attachments/20090603/db43e576/attachment-0001.pgp