On Thu, Jan 30, 2003 at 05:12:56PM +0100, Roberto Nibali wrote:
Hi,
I'm conducting syslog-ng performance tests. I've deployed a central loghost environment and three clients trying to send messages which should end up on the central loghost with as few losses as possible. The whole thing is currently done over UDP and the clients have either syslogd or syslog-ng as daemons.
If I deploy syslog-ng as a client to send messages from /dev/log via UDP to the central loghost I experience a rather high message loss starting on a threshold of about 1000 messages sent in the least amount of time possible. If I do the same thing with syslogd the threshold is up to 35000 from the same machine. That means I can send 35000 messages to /dev/log as fast as possible and they still get reliably transferred to the central loghost and written to disk there.
A little stunned over this strange occurrance I've started to narrow the problem down and I ended up with following question: Why does syslog-ng not send a UDP packet per message sent via syslog() and an ending '\n'? And is there a mean to configure this odd behaviour? To illustrate what I mean I provide you with two sample dumps from a tcpdump session where I sent 3 messages. The first one is the tcpdump snipped of syslogd processing the messages and the second one is the result when syslog-ng processes the logs:
Both TCP and UDP use the same code to send messages in 1.5.x and as it seems syslog-ng piggybacks messages together when writing messages. This is correct for TCP but incorrect for UDP. The culprit is libol/src/pkt_buffer.c which uses stream based buffering in the 0.3 branch while it used a packet based one in 0.2 As there are more than one message in a single UDP packet the packet can grow bigger than your MTU, thus packet fragmentation is done. Both can increase message loss (more than one message in a single packet, fragmentation) thus this is definitely a bug. I'll look into fixing this.
Another question: Why is there no message sent when a buffer reaches its maximum size? This would help debugging and tuning a lot IMHO. Sorry if this has been discussed before.
Because this message itself would fill your buffers. Instead syslog-ng reports the number of dropped messages in a separate message sent once every ten minutes (default). -- Bazsi PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1