[syslog-ng] Ubuntu Precise -ng filling out buffer, dropping messages

[Chaman Chakalaka]

Hi there,

I'm trying to process ~800 UDP messages second, which I don't think is much.  The current setup worked fine in Ubuntu 10.04 (Lucid) and syslog-ng 2.6 (I believe).  I'm running into what I believe is receive buffer problems on Ubuntu Server 12.04 (Precise) w/ ng 3.XX

Via tcpdump, I can attest all messages are getting to the server.  I'm listening to port 514 UDP.  From syslog-ng stats, I don't see any "dropped" statistics (always 0), but I know for a fact that when the message load is more than 150, I start seeing message loss.  All messages received are sent to files on local disk after matching some filters.  I have tested this with loggen and no filters -- more messages are sent than written to file.

netstat -su shows increasing number of receive errors when going beyond 150 msgs per second.  It also shows the receive queue hover in the 20000's.  I have tried setting the so_rcvbuf to a relatively large value as described in the admin guide (2 097 152 bytes).  The default net.core.rmem_max on ubuntu server 12.04 is 8388608.  I have played with this number, up and down with no luck -- messages are still lost.

I have spent hours reading documentation, doing packet captures, playing with values in -ng and the host, etc, but I'm at a loss.  Any hints as to what I should look into?

And more direct questions, 
Will so_rcvbuf declared on the src modify the system's rmem_max or do I have to do it on both?  
Why is it recommended in the admin guide to not modify the rmem_default?

Thanks any advice would be helpful.


 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20140429/6b88876a/attachment.htm