[syslog-ng]performance test questions

[Roberto Nibali]

Hello,

> I also faced a problem with udp message loss and performed some tests of syslog-ng
> with attemts to reduce them. So here are several suggestions..

Thanks.

> First, i noticed that with high udp message rates number of udp packets
> were dropped due to full socket buffers.

Sometimes as my tests reveiled so far.

> This may be checked with netstat's per protocol statistics (netstat -s),
> Increasing socket rx buffer to 3 MB with setsockopt SO_RCVBUF option seems to help.

I'll try.

> though it does not completely stops packet drops

I think this is impossible anyway ;).

> Second, if resolving is required, either syslog-ng dns cache should be enabled,
> or local caching name server should be installed to speed up resolving, as slow
> resolving leads to udp messages loss.

Yes, absolutely and I've done that too.

> I also turned off all usertty destinations because they seemed to slow down syslog-ng,
> constantly reading /var/run/utmp.

They are off for my tests.

> Unfortunately i could not get rid of udp packets drops comletely,
> when peak message rates jumps above 4000 msgs/sec number of packets are being dropped :(

That's what I'm seeing roughly but it so dependant on the machine's state as I 
have seldomly seen before. And simple variations in configuration or test 
conducts sometimes have a tremendous impact on the test results (almost chaotic 
behaviour :))

> Does anybody have ideas about how to reduce loss further?

Under Linux I tried tweaking some values in /proc/sys/net/core/* but to no avail 
so far. Also it occurs to me that most of those entries only affect TCP sockets. 
I'm reading up on the kernel code to see if there is anything else that might help.

Another thing of course could be fair scheduling and rate limiting, though I'm 
not entirely sure if such changes have a significant impact on the actual 
problem of lost messages.

Thanks and best regards,
Roberto Nibali, ratz
-- 
echo '[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq' | dc