[syslog-ng]Massive lossage with syslog-ng

syslog-ng@lists.balabit.hu syslog-ng@lists.balabit.hu
Thu, 18 Mar 2004 11:35:41 -0800


My apologies if this has been discussed, as I haven't been able to find
anything useful via google or the FM...

Syoposis:

Syslog-ng drops something on the order of 90% of the logs remotely flung
at it.

Detail:

I'm using syslog-ng 1.5.15 from the Debian stable package archive.

I've been tasked with setting up two remote log servers for my employer;
both log servers have fairly beefy IDE raids (IOZone gives me an
unbuffered write speed of about 40M/sec), and as far as I can tell with 
vmstat(8) and Our Friend top(1), syslog-ng isn't running into any I/O
bottlenecks.  The systems have insane CPUs (Athlon XP 2000) and 512M RAM
a pop, which considering their intended tasks (syslog and serial console
server), should be more than adequate.

I've set up several systems to push their logs onto the log server; a
Solaris 8 (with native syslog) box, a Debian Linux box (native syslog
again), and a Debian Linux box using syslog-ng.

Yet syslog-ng seems to dump between 75% and 90% of all the logs handed
to it on the floor.  Several of the systems we wish to have logging to
syslog will throw out about a thousand lines within a second or two, and
this is where most of our problems come in.

I've tried both TCP and UDP; TCP barely seems to work at all --  I can
throw a thousand lines in on one end, see them in the logfile, and see
one line of output out on the other side.  I've futzed a bit with the
FIFO size and the gc_idle/gc_busy numbers, but the latter two are 
more-or-less undocumented in the manual, so I have no idea what they 
really do (no units or anything), and the code regarding them is a bit 
too complex for me to read.

I've been googling and chomping upon the syslog-ng documentation for
about the past two weeks, and have yet to find a solution.

Any suggestions?  An FM for me to read?  A USENET article I missed?

Thanks-in-advance!

-- 
Don Werve <donw@examen.com> (Unix System Administrator)

Yorn desh born, der ritt de gitt der gue,
Orn desh, dee born desh, de umn bork! bork! bork!