[syslog-ng] Lost messages

[Balazs Scheidler]

On Fri, 2007-12-21 at 10:56 +0100, Andrew Séguin wrote:
> Thank you for the quick reply.
> 
> Sorry for not mention, since are starting this now, we used the latest
> version, so syslog-ng 2.0.6...
> 
> We looked at the stats (set to 10s), and saw the following:
> 
> $ ./syslog-ng -v -e -F -f /tmp/syslog- ng.conf.solaris
> syslog-ng starting up; version='2.0.6'

> Log statistics; processed='center(queued)=578629',
> processed='center(received)=578630',
> processed='destination(all)=578629', processed='source(local)=578630' 
> 
> Our test program (same host, a Sun Fire v440 running Solaris 10) had
> tried to log 1000000 messages. I guess this means the loss is before
> it reached Syslog-ng?
> 
> Do you have any idea how we can get around this? The more
> logging-intensive applications, have their sources available to us, a
> few however don't, so maybe combining some other source with /dev/log
> could help? 

I didn't recognize until now that the STREAMS device behind /dev/log can
actually lose messages.

Hm.. I did some research:

     Normally, putmsg() will block if the stream write  queue  is
     full   due  to  internal  flow control conditions. For high-
     priority messages,  putmsg() does not block on  this  condi-
     tion.  For other messages,  putmsg() does not block when the
     write queue is full and   O_NDELAY  or  O_NONBLOCK  is  set.
     Instead, it fails and sets  errno to EAGAIN.

     The putmsg()  or  putpmsg()  function  also  blocks,  unless
     prevented  by  lack  of internal resources, waiting  for the
     availability of message blocks in the stream, regardless  of
     priority   or whether O_NDELAY or O_NONBLOCK has been speci-
     fied. No partial message is sent.

This essentially says, that putmsg() in the calling application should
block if the STREAMS queue is full, that is: it waits for syslog-ng to
process messages.

How do you generate messages? What's the message rate (e.g. msgs/sec)
when this message loss occurs?

-- 
Bazsi