Out of curiosity, how many messages per second was the stock syslogd able to process with minimal loss? What method did you employ to determine loss? I am setting up a similar logging solution with NG using the db-parser module which takes considerable CPU. I plan on using Cisco server load balancing to round-robin load balance on N number of syslog nodes to achieve zero loss, but I'm concerned that detecting loss will be difficult. --Martin On Sat, May 30, 2009 at 12:43 PM, Jan Schaumann <jschauma@netmeister.org>wrote:
Sandor Geller <Sandor.Geller@morganstanley.com> wrote:
This is somewhat expected as syslog-ng parses incoming messages. So my I guess is that syslog-ng can't drain fast enough the receive buffer, and the kernel simply drops messages not fitting in the buffer.
Exactly.
It would be good to know whether the source side or the destination side is the limiting factor. As you're using local files myguess is the former.
I'm quite sure the source side is the problem. Ie, I/O to the file on disk ought to be reasonably fast (otherwise stock syslogd would have the same problems). As you noted, the additional processing that syslog-ng does for every message it receives seems to cause it to not be able to process them fast enough to drain the buffers.
flags(flow-control)
in the log definition.
AFAIK with files/ UDP flow-control is a no-op.
Ah, good to know.
Unfortunately this can't happen. You can use the 'no-parse' option to skip initial parsing the messages which could improve performance. This means you can't use the template above as the variables won't get defined.
I'll have to give that a try, if only to determine what, if any, performance difference it causes.
Generally when it comes to parsing then syslog-ng could be CPU-limited. In this case you should consider deploying multiple syslog servers, and share the load. Ideally flow-controlling could be turned on the client side as well (using TCP).
Yes, those are the long-term plans. :-) Well, we can't switch all clients to TCP, since many of them are network/storage devices etc. only capable of logging via UDP.
For the time being, though, I need to lay the ground work of getting syslog-ng as a suitable replacement for the stock syslogd used on our servers.
Thanks for your help. I'll keep poking at this...
-Jan
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html