[syslog-ng]Massive lossage with syslog-ng

[Balazs Scheidler]

2004-03-18, cs keltezéssel 20:35-kor donw@examen.com ezt írta:
> My apologies if this has been discussed, as I haven't been able to find
> anything useful via google or the FM...
> 
> Syoposis:
> 
> Syslog-ng drops something on the order of 90% of the logs remotely flung
> at it.

you should first try to identify where the message is dropped. it is
possible that the packet is not received by syslog-ng at all, because
the local IP stack drops it (for instance because the local receive
buffer is full) (the UDP case) You could check this by comparing a
tcpdump and an strace dump of the same time interval and check whether
each incoming packet has its associated recvfrom() system call.

TCP should not drop messages on the network but messages might still be
lost on the sending side, syslog-ng itself is not flow controlled, once
the output pipe send buffer is full, it will start dropping messages.
You could check the "STATS" message syslog-ng is producing, it reports
all messages dropped within syslog-ng.

You could also update your syslog-ng, the version you are using is quite
outdated, you can find binaries for woody in our ZorpOS repository.
(check: http://www.balabit.com/downloads/zorp/zorp-os/pool/s/syslog-ng/)


> 
> I've set up several systems to push their logs onto the log server; a
> Solaris 8 (with native syslog) box, a Debian Linux box (native syslog
> again), and a Debian Linux box using syslog-ng.
> 
> Yet syslog-ng seems to dump between 75% and 90% of all the logs handed
> to it on the floor.  Several of the systems we wish to have logging to
> syslog will throw out about a thousand lines within a second or two, and
> this is where most of our problems come in.
> 
> I've tried both TCP and UDP; TCP barely seems to work at all --  I can
> throw a thousand lines in on one end, see them in the logfile, and see
> one line of output out on the other side.  I've futzed a bit with the
> FIFO size and the gc_idle/gc_busy numbers, but the latter two are 
> more-or-less undocumented in the manual, so I have no idea what they 
> really do (no units or anything), and the code regarding them is a bit 
> too complex for me to read.
> 
> I've been googling and chomping upon the syslog-ng documentation for
> about the past two weeks, and have yet to find a solution.
-- 
Bazsi
PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1