[syslog-ng] Syslog-ng 1.6.9 just stops...

[Balazs Scheidler]

On Wed, 2006-03-01 at 10:01 -0500, Andreoli, Tony A. USNUNK NAVAIR
B1490, R215 wrote:
> I'm using 1.6.9 (upgraded from 1.6.6 because I was seeing the same
> problem).  I have it running on 8 different servers at different
> locations, some are SMP, some aren't.  On these hosts, we have
> anywhere from 2 to 14 devices logging to the servers, some via
> 514/tcp, others via 514/udp.  All of the loggers typically sit with a
> load average < 1 (usually not even registering), and a cpu idle of
> 99%.  7 of these remote loggers also log to our local machine, but
> only 5 lines every 2 minutes (for stats).  
> 
> What I've noticed (and I've seen this on all of them at one time or
> another), is that syslog-ng just stops.  ps shows it running, but the
> log file (/logs/messages) never changes.  If I tcpdump on the
> interface that it's listening on, I see traffic, and it seems that the
> act of tcpdumping causes the log file to start to grow again, then a
> little while later, it may stop again.  It's sporadic though, on one
> of my systems, it hasn't done it in over 2 months, on another, it's
> done it 3 times today.
> 
> I've pulled out my last hair and still haven't come any closer to a
> solution.  I've recompiled the source, loaded 3 different versions,
> etc.  The only thing common is that all of these systems are running
> RHEL3.

Don't you happen to read /proc/kmsg by both syslog-ng and klogd ? That
is a known bad situation and the symptoms are exactly what you describe.
(poll indicates readability but by the time syslog-ng gets to read the
file the data has already been read)

This is documented in the FAQ as well.

-- 
Bazsi