[syslog-ng] Re: Reliable tcp logging

[Peter Daum]

Roberto Nibali wrote:
>> Well, yes, it is exactly the same issue and it is indeed only one line
>> that gets lost (which in my case, where typically every host sends about
>> 1 line/hour does not really make a difference).
> 
> 
> You mean 1 line/hour that is lost, right?

I guess, my description was ambiguous.
My problem is _not_ excessive packet loss because syslog-ng couldn't handle
the volume but really just the contrary: Per host there is typically maybe
than one line/hour and if that line gets lost, this is a significant
percentage.

I have a "classical" loghost where all kinds of machinery sends their log
messages to via udp, That loghost runs syslog-ng and sorts all the messages
neatly into different files. I didn't systematically investigate, but I
don't have any reason to believe that much gets lost. Because everything
works so nicely (I switched to syslog-ng fairly recently and am very
thrilled; my thanks to everybody who contributed to it:-), I decided to
extend the central logging:

There is a bunch of server machines, which in maintain their own local
logfiles and in general this is fine. What I am trying to do now, is
collect (in addition to the "normal" logging) everything that is important
enough to require immediate attention in one location at the loghost.

For this, I switched completely to syslog-ng and configured all boxes to
forward everything beyond a certain priority via tcp to the loghost.
Because I am still fine-tuning the setup (weeding out messages that are
sent with a far-to-high priority), I occasionally have to reload the
configuration (which also results in all network connections being
dropped). This is where I discovered, that if the loghost is restarted
for any reason, it takes up to 2 hours for the clients to notice and if
they try to send anything during this time it is lost. In my case this
is fatal because the hole idea is to normally only watch one log file
and rely on everything important showing up there.

I guess, for me currently the best option would be to switch to udp
instead (maybe on a different port to keep the important stuff separate
from printers telling about being out of paper), or get really daring
and try 1.9.x ...

Regards and Thanks,
                      Peter Daum