On Fri, 2012-03-02 at 10:40 +0100, Daniel Neubacher wrote:
Hello there,
I’ve started playing around with syslog-ng 3.3.4 ose a few days ago but I’m still experiencing some trouble. First of all we want to use syslog-ng to send all of our logs via udp to a central syslog server. This includes of course syslogs, apache logs and custom generated applogs. These logs are generated from 400 clients and produces a minimum of 300 mio. log lines a day.
The problem is really simple: I’m losing log lines :P Most of the time everything goes well but when the logs are peaking high 1-5% logs are getting lost.
Last night the stats of the server and a client said 0 drops but when I counted the lines I found lost lines. The server has 24g ram & 8 cores and I can rule out a network problem for sure.
So now to my questions, has anyone else an idea where I can tweak my cfg or where I have to look to find more clues? Is tcp the only way to get around it?
I’ve attached my syslog server cfg. The so_rcvbuf buffer is the same size as the os net.core.rmem settings. And as described in the various balabit blog posts I played around with log_fetch_limit and flush_lines already.
I've received reports, that syslog-ng's latency over UDP sources became worse with the threaded architecture and threaded(yes) is specified. One possible workaround until I get around to look into the issue a bit deeper is to disable the global threaded() option and enable the "threaded" flag on all non-udp sources and all destinations. This should basically be the same, except that udp traffic will be processed in the main thread, which effectively reduces the latency for the udp source. I saw that you rolled out tcp() already, but this information could help someone else trying to tackle the same issue in the future. -- Bazsi