[syslog-ng]Losing 25% of messages using UDP

[Bill Graham]

Balazs Scheidler wrote:

>On Thu, Apr 10, 2003 at 06:49:56PM -0700, Bill Graham wrote:
>  
>
>>Balazs Scheidler wrote:
>>    
>>
>>>syslog-ng uses a poll() loop to check whether a given source (e.g. UDP
>>>socket) is readable and once it is, it issues one single recvfrom() and 
>>>then
>>>returns to the mainloop.
>>>
>>>So your host is not fast enough to keep up with the message rate (at least
>>>not when using one message/poll loop)
>>>
>>>The following options are available:
>>>1) upgrade the hw
>>>2) increase the default UDP sockbuf size to keep up with bursts
>>>3) implementat issuing several recvfrom() when poll() indicates readability
>>>
>>>The 3) option involves adding a loop in sources.c:do_read_line() function
>>>which would call recvfrom() as long as it returns that nothing is 
>>>available.
>>>Maybe an upper limit to avoid starving other sources would be needed (say
>>>read until anything is available but no more than 10 messages)
>>>
>>>      
>>>
>>I increased my default UDP sockbuf  and it seems to have solved the problem.
>>    
>>
>
>Although it might solve the problem for the bursts you are testing with you
>might run into limits again with greater loads.
>
I agree that I might run into the problems again, but I have tested the 
system with bursts of 500 messages and it seems to handle the load.  
Upgrading the system probably wouldn't solve the problem and would most 
likely be a waste of hardware.  I am running on a E250 w/2 450MHz 
processors and 4Gb of RAM.  I am also not experienced enough of a 
programmer to rewrite the program.  Is this something that you could 
address in a future release of the software?

Bill