[syslog-ng] Tests using loggen - not receiving all the packets

Balazs Scheidler bazsi at balabit.hu
Wed Apr 14 19:36:59 CEST 2010


On Wed, 2010-04-14 at 17:30 +0200, Zoltán Pallagi wrote:
> Clayton Dukes wrote: 
> > Excellent link, thanks!
> > That does seem to be the problem, however, if I set the buffer all the
> > way up to 1G using:
> > sysctl -w net.core.rmem_max=1073741824
> > 
> > 
> > Then I'm still dropping messages when using a test rate of 6kmps:
> > 
> > # ./loggen -r 6000 -D -I 10 127.0.0.1 514
> > average rate = 6526.63 msg/sec, count=65272, time=10.008, msg
> > size=256, bandwidth=1631.66 kB/sec
> > 
> > # wc -l /tmp/logs
> > 62933 /tmp/logs
> > 
> > Is there a recommendation on what the buffer should be set to for high
> > insertion rates?
> > My test server has 8G of memory, but I can give it more (up to 24G).
> > 
> > Also, note that this is a VMWare ESXi server - might that have
> > something to do with it?
> > 
> > 
> >   
> 
> It's interesting. I tried it with rmem_max=1MB, and worked without
> dropped messages (my machine is  Intel(R) Core(TM)2 CPU          6300
> @ 1.86GHz, with ubuntu)
> 
> root at thor:/opt/syslog-ng/bin# ./loggen -r 6000 -V -D -I 30 127.0.0.1
> 2222
> average rate = 5991.87 msg/sec, count=179757, time=30.001, (last) msg
> size=256, bandwidth=1497.97 kB/sec
> 
> root at thor:/var/log# wc -l test.log 
> 179757 test.log
> 
> root at thor:/var/log# cat /proc/sys/net/core/rmem_default
> 1048576
> 
> But if I set the rmem_max to 1MB, I have also dropped packages, If I
> set the rmem_default it works... (I don't know why, I am not an
> udp-kernel magus). 
> Will you try if you set rmem_default instead of rmem_max?

quoting socket(7) on Linux

       rmem_default
              contains the default setting in bytes of the socket receive buffer.

       rmem_max
              contains the maximum socket receive buffer size in bytes which a user may set by using the SO_RCVBUF socket option.


syslog-ng allows you to change the socket buffer given rmem_max allows
that using the so_rcvbuf() option for the given source.

Changing rmem_default at the system scope might be a bit of an overkill,
as _all_ udp sockets would get this large socket buffers.

And socket buffers are in non-pageable memory, don't forget that.

-- 
Bazsi



More information about the syslog-ng mailing list