[syslog-ng]losing messages using syslog-ng-1.4.7 & libol-0.2.20

[Balazs Scheidler]

On Thu, Oct 26, 2000 at 04:18:23PM +0000, matthew.copeland@honeywell.com wrote:
> > 
> > You're almost certainly overflowing the internal queue in either the
> > server or the clients or both.  Try logging the messages on the clients to
> > a file as well as the network, and see if the local syslog-ngs are
> > dropping messages before the server has any chance.
> 
> 	I can give this a try on the server side, but on the client side I
> have a custom program that is doing the sending.  It could be that the
> buffers are getting overflown there, but I am finding that I am having a
> hard time telling.  Also, if it were on the client side, then in theory, I
> should get one of the attempts that gets all of the messages as the
> os scheduler schedules the threads differently to send, so I am thinking
> it must be on the server side somewhere.

I tried to run your program, but it couldn't create threads over 256, I've
recompiled my kernel, but didn't have enough time to get to it again.

> > You can put some printf statements into the libol code to trace what is
> > happening with each log entry and the output queue.  I did this to mine
> > and I can see in the output that syslog-ng sometimes (often) simply throws
> > messages away.  The interesting parts are in pkt_buffer.c and queue.c in
> > libol:
> > 
> >         if (self->queue_size == self->queue_max) {
> >                 /* fifo full */           <== oops
> >                 ol_string_free(string);   <== this tosses your message
> >                 return ST_FAIL | ST_OK;   <== return code is ignored
> >         }
> 
> 
> Okay I set log_fifo_size to be something like 72,000 does that not
> integrate well with queue_max?  I will give this a try though and see if
> this is where I am losing messages.

queue_max is equal to log_fifo_size here.

-- 
Bazsi
PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD 804E C82C 8EB1
     url: http://www.balabit.hu/pgpkey.txt