[syslog-ng]performance test questions

[Roberto Nibali]

> Balazs has found some problems he's looking into, but there's something
> left for you to do. You need to establish a baseline for the maximum
> throughput your OS and network can send over UDP. If you setup a

Well, I haven't explained my whole test setup in my first email. I do tests like 
that too, although there is little point in doing such tests with the current 
result. As long as the percentage of packet loss seen for the daemons differ in 
such a significant way it is an exercise in futility and thus a waste of time.

Once those initial problems get sorted out, I will conduct raw UDP throughput 
tests to compare numbers. Another thing is that if you add the metric of UDP 
'client - server' throughput performance you add another dozen or so knobs 
representing parameters. To be able to control your test conducts and generate 
appropriate statements based on observation less parameters yield more accuracy.

> listener on a UDP port that simply discarded all input (instead of
> syslog-ng) and send UDP packets with little or no payload, you may still
> get all kinds of packet loss.

I do count packet loss with a kernel patch I've done to the iproute2 framework, 
plus I do vary buffer sizes of TX/RX queues, plus I've a fallback counter done 
with ipchains/iptables/pf rules and I permanently have an xterm with a tcpdump 
running. The problem is that tcpdump doesn't get all packets, so dropped packets 
differ between what tcpdump and iproute2 are seeing.

> The reason for this is because it's possible to overflow your UDP send
> buffer before anything is even written to the network. Your message loss
> may primarily be on the sending system, and have little to do with
> syslog-ng.

Absolutely correct. With Linux however, you are provided with very useful 
network related statistics under /proc/net/*. Using the latest iproute2 and 
iputils package from Alexey you even get human readable output in realtime :).

> <URL:http://cert.uni-stuttgart.de/archive/loganalysis/2002/01/msg00049.html>

Very interesting reading, thanks.

Best regards,
Roberto Nibali, ratz
-- 
echo '[q]sa[ln0=aln256%Pln256/snlbx]sb3135071790101768542287578439snlbxq' | dc