Out of curiosity, how many messages per second was the stock syslogd able to process with minimal loss? What method did you employ to determine loss? I am setting up a similar logging solution with NG using the db-parser module which takes considerable CPU. I plan on using Cisco server load balancing to round-robin load balance on N number of syslog nodes to achieve zero loss, but I'm concerned that detecting loss will be difficult.<br>
<br>--Martin<br><br><div class="gmail_quote">On Sat, May 30, 2009 at 12:43 PM, Jan Schaumann <span dir="ltr"><<a href="mailto:jschauma@netmeister.org">jschauma@netmeister.org</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im">Sandor Geller <<a href="mailto:Sandor.Geller@morganstanley.com">Sandor.Geller@morganstanley.com</a>> wrote:<br>
<br>
> This is somewhat expected as syslog-ng parses incoming messages. So my<br>
> I guess is that syslog-ng can't drain fast enough the receive buffer,<br>
> and the kernel simply drops messages not fitting in the buffer.<br>
<br>
</div>Exactly.<br>
<div class="im"><br>
> It would be good to know whether the source side or the destination<br>
> side is the limiting factor. As you're using local files myguess is<br>
> the former.<br>
<br>
</div>I'm quite sure the source side is the problem. Ie, I/O to the file on<br>
disk ought to be reasonably fast (otherwise stock syslogd would have the<br>
same problems). As you noted, the additional processing that syslog-ng<br>
does for every message it receives seems to cause it to not be able to<br>
process them fast enough to drain the buffers.<br>
<div class="im"><br>
<br>
> > flags(flow-control)<br>
> ><br>
> > in the log definition.<br>
><br>
> AFAIK with files/ UDP flow-control is a no-op.<br>
<br>
</div>Ah, good to know.<br>
<div class="im"><br>
> Unfortunately this can't happen. You can use the 'no-parse' option to<br>
> skip initial parsing the messages which could improve performance.<br>
> This means you can't use the template above as the variables won't get<br>
> defined.<br>
<br>
</div>I'll have to give that a try, if only to determine what, if any,<br>
performance difference it causes.<br>
<div class="im"><br>
> Generally when it comes to parsing then syslog-ng could be<br>
> CPU-limited. In this case you should consider deploying multiple<br>
> syslog servers, and share the load. Ideally flow-controlling could be<br>
> turned on the client side as well (using TCP).<br>
<br>
</div>Yes, those are the long-term plans. :-) Well, we can't switch all<br>
clients to TCP, since many of them are network/storage devices etc. only<br>
capable of logging via UDP.<br>
<br>
For the time being, though, I need to lay the ground work of getting<br>
syslog-ng as a suitable replacement for the stock syslogd used on our<br>
servers.<br>
<br>
Thanks for your help. I'll keep poking at this...<br>
<font color="#888888"><br>
-Jan<br>
</font><br>______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.campin.net/syslog-ng/faq.html" target="_blank">http://www.campin.net/syslog-ng/faq.html</a><br>
<br>
<br></blockquote></div><br>