[syslog-ng] UDP packet loss with syslog-ng

[Matthew Hall]

This is not at all a surprise to me. I have nothing but problems with 
RHEL. In this case SLES 11 was released in 2009 and RHEL 5 was released 
in 2007. Not to mention Redhat applies a lot of idiotic patches to their 
code.

I am guessing the kernel and scheduling behavior are different so a 
different buffer size is required. Their could also be differences in 
network adapter polling versus interrupt processing, etc.

Without trying it under different situations and running oprofile or 
systemtap etc. it's hard to say in detail.

Matthew.

On Sat, Oct 16, 2010 at 03:02:33PM -0500, Martin Holste wrote:
> Glad to hear you got things working, but I don't understand why this
> isn't an issue on SLES 11 with default buffer sizes (131071 bytes) in
> which I can get > 15k mps with 0 drops.
> 
> On Sat, Oct 16, 2010 at 1:10 PM, Lars Kellogg-Stedman <lars at oddbit.com> wrote:
> >> Hmm. the numbers you are seeing are indeed low, with sufficient buffer
> >> sizes I could get up to the 20k message/sec range with syslog-ng...
> >
> > It's better now, having adjusted the buffer sizes way up.  I'd like to
> > recommend a change to the documenation.  In section 7:
> >
> >  http://www.balabit.com/sites/default/files/documents/syslog-ng-admin-guide_en.html/handling_large_load.html
> >
> > The issue of buffer size is addressed like this:
> >
> > "This section provides tips on optimizing the performance of
> > syslog-ng. Optimizing the performance is important for syslog-ng hosts
> > that handle large traffic... When receiving lots of messages using the
> > UDP protocol, increase the size of the UDP receive buffer on the
> > syslog-ng hosts."
> >
> > I would suggest that with the default Linux kernel values for UDP
> > receive buffer size, adjusting the UDP receive buffer size is
> > necessary to get performance above "crappy".  That is, this isn't just
> > a necessity for "high volume" sites; it should probably be a
> > recommended practice for anyone planning on accepting UDP syslog
> > messages on a Linux host.  Making this more prominent in the
> > documentation might save a lot of people from the rude surprise that
> > comes with the default buffer sizes.
> >
> > Things are running much better now having made these changes.  I'm
> > going to write up the performance test I did in a little more detail
> > and stick it online somewhere, hopefully saving someone else a little
> > bit of time in the future.
> > ______________________________________________________________________________
> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> > FAQ: http://www.campin.net/syslog-ng/faq.html
> >
> >
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>