[syslog-ng]performance test questions

[Dmitry Frolov]

Hi,

I also faced a problem with udp message loss and performed some tests of syslog-ng
with attemts to reduce them. So here are several suggestions..
First, i noticed that with high udp message rates number of udp packets
were dropped due to full socket buffers.
This may be checked with netstat's per protocol statistics (netstat -s),
Increasing socket rx buffer to 3 MB with setsockopt SO_RCVBUF option seems to help.
though it does not completely stops packet drops
Second, if resolving is required, either syslog-ng dns cache should be enabled,
or local caching name server should be installed to speed up resolving, as slow
resolving leads to udp messages loss.
I also turned off all usertty destinations because they seemed to slow down syslog-ng,
constantly reading /var/run/utmp.
Unfortunately i could not get rid of udp packets drops comletely,
when peak message rates jumps above 4000 msgs/sec number of packets are being dropped :(
Does anybody have ideas about how to reduce loss further?

On Thu, Feb 20, 2003 at 04:47:53PM +0100, Roberto Nibali wrote:
> Hi,
> 
> >no, your feedback is appreciated.
> 
> Ok, meanwhile I've conducted some tests with syslog-ng-1.5.26 and 
> libol-0.3.9 plus a hand-applied templates patch. I've written a little test 
> tool to send log messages to /dev/log where they will be fetched by 
> syslog[d|-ng] and transported via UDP or in case of syslog-ng over UDP/TCP 
> to a central syslog-ng server. The tests were all done over a 10Mbit/s 
> switched network with no other traffic. The hosts are all running a 2.2.x 
> kernel and tools have been compiled with glibc-2.1.3.
> 
> I was interested in how many log messages (burst mode, meaning that there 
> is no usleep between sending messages off via syslog()) one can reliably 
> send over the net and store on the disk depending on the level of enabled 
> MACRO expansion. The results during the tests were quite instable so I had 
> to do a lot of repetitions to get a useful mean per test case.
> 
> I do not claim full correctness of these results but they sure give you an 
> impression on the impact of using UDP vs. TCP or enabling MACRO expansion 
> on different levels.
> 

-- 
Dmitry Frolov, Zenon N.S.P. (095) 250-4629, http://www.zenon.net/