[syslog-ng] syslog-ng blocking on read() from /proc/kmsg

[John Morrissey]

I'm running the Debian 1.6.9-1 package with the 'read from each file
descriptor n times or until EAGAIN' patch backported. There seems to be a
problem with this behavior when it's applied to /proc/kmsg. Here's the
source we use:

source s_kmsg {
        file("/proc/kmsg" log_prefix("kernel: "));
};

What happens is:

1. syslog-ng calls poll(2) and waits.
2. Data appears on /proc/kmsg and causes poll() to return.
3. syslog-ng read(2)s from /proc/kmsg, grabbing some data.
4. Since it hasn't received EAGAIN or read n times from this file
   descriptor, syslog-ng happily read()s again. Since it's exhausted
   the available data, the read() blocks. Forever.
5. Meanwhile, other syslog traffic to /dev/log piles up. Eventually,
   whatever buffer there is on Unix domain sockets fills up and causes
   other applications on the machine to block, waiting to syslog.
6. The box eventually runs out of file descriptors since other processes
   (cron is a big culprit because it runs jobs so often) are piling up,
   holding onto file descriptors and blocking on writes to /dev/log.

The blocking behavior on /proc/kmsg is interesting, since if I'm reading the
source correctly, file sources are set O_NONBLOCK:

affile.c:
  152 static int do_init_affile_source(struct log_handler *c, struct syslog_con
  153 {
  154     CAST(affile_source, self, c);
  155     int fd, flags;
  156
  157     if (self->flags & AFFILE_PIPE)
  158         flags = O_RDWR | O_NOCTTY | O_NONBLOCK | O_LARGEFILE;
  159     else
  160         flags = O_RDONLY | O_NOCTTY | O_NONBLOCK | O_LARGEFILE;

Any thoughts?

thanks,
john
-- 
John Morrissey          _o            /\         ----  __o
jwm at horde.net        _-< \_          /  \       ----  <  \,
www.horde.net/    __(_)/_(_)________/    \_______(_) /_(_)__