[syslog-ng]TCP transport reliability

[Jeffrey W. Baker]

On Thu, 4 Jan 2001, Balazs Scheidler wrote:

> Hello,
>
> > > I'm testing syslog-ng with TCP transport, and I'm a little surprised-
> > > perhaps I'm doing something wrong? I'm finding severe message loss,
> > > which I had imagined TCP would eliminate.
> >
> > I've reported the same thing on this list in the past.  Indeed, my
> > machines can even lose messages when the input is a pipe and the output is
> > a local file!
>
> And I replied that you are _not_ correct. Several times. Please stop
> spreading misinformation.

Repeating yourself doesn't make you right.  Look at the code again:


         if (self->queue_size == self->queue_max) {
                 /* fifo full */
                 ol_string_free(string);
                 return ST_FAIL | ST_OK;
         }

What do you think happens when this code is hit?  The log message is
discarded and lost.  It cannot be read any other way.  When this block of
code is executed, messages are lost.  Execution tracing shows that this
block of code is sometimes executed.  Thus, messages will sometimes be
lost.  QED.

Your repeated assertions are no solution.  In practice, syslog-ng drops
log messages.  I have seen it with my own eyes, and I have proved it
logically above.  You have not given me any reason to believe otherwise.
Your suggestion to raise the fifo size does not work for fast,
steady-state log message generation.  I can raise the fifo size to
1,000,000 messages, and I get a syslog-ng process that consumes all
available memory and still drops messages.

-jwb