<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
<meta content="text/html;charset=ISO-8859-1" http-equiv="Content-Type">
</head>
<body bgcolor="#ffffff" text="#0050d0">
<font size="-1"><font face="Helvetica, Arial, sans-serif">Ok, so
finally got time to look at this again.<br>
I tried the time_sleep option, and it was a horrible failure. With it
enabled syslog-ng started losing about 98% of all incoming lines (and
no, thats not an exaggeration). I'm guessing that time_sleep does not
play well with udp as thats what incoming data is being sent over.<br>
However I do have the master-slave multi-process thing going and its
working really good. I was able to put time_sleep on the child
processes (the one doing regex matches), and it dropped their cpu
utilization from around 40% to about 20% (master process uses tcp to
talk to slave processes, so no drops). Another thing is that when I
tried using the syslog protocol to talk to the child processes, the
slaves were terminating the connection within seconds of being
established. I poked and prodded and could not get this to work without
constantly dropping the connection, so I had to switch back to plain
tcp.<br>
<br>
Anyway, the attached config is what it looked like when I had all
regexes run within a single process (the config that was utilizing over
90% cpu).<br>
<br>
<br>
</font></font><br>
Sent: Thursday, March 18, 2010 10:56:16 PM<br>
From: Jan Schaumann <a class="moz-txt-link-rfc2396E" href="mailto:jschauma@netmeister.org"><jschauma@netmeister.org></a><br>
To: <a class="moz-txt-link-abbreviated" href="mailto:syslog-ng@lists.balabit.hu">syslog-ng@lists.balabit.hu</a> <br>
Subject: Re: [syslog-ng] log failback groups
<blockquote cite="mid:20100319045615.GB28121@netmeister.org" type="cite">
<pre wrap="">Martin Holste <a class="moz-txt-link-rfc2396E" href="mailto:mcholste@gmail.com"><mcholste@gmail.com></a> wrote:
</pre>
<blockquote type="cite">
<pre wrap="">How many messages per second is the system attempting to handle? I'm
very surprised that you're seeing that level of utilization. In our
setup we've never had a problem pushing up through 30,000 messages per
second written to disk with Syslog-NG in production, and I've pushed
more than 70,000 per second in development.
</pre>
</blockquote>
<pre wrap=""><!---->
Could you provide your configuration for these systems (including
sysctls or kernel tunables etc.)? I've so far not been able to get my
systems to accept and process (without any regex matching) more than
approximagely 25K - 30K UDP messages/s.
-Jan
</pre>
<pre wrap="">
<hr size="4" width="90%">
______________________________________________________________________________
Member info: <a class="moz-txt-link-freetext" href="https://lists.balabit.hu/mailman/listinfo/syslog-ng">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a>
Documentation: <a class="moz-txt-link-freetext" href="http://www.balabit.com/support/documentation/?product=syslog-ng">http://www.balabit.com/support/documentation/?product=syslog-ng</a>
FAQ: <a class="moz-txt-link-freetext" href="http://www.campin.net/syslog-ng/faq.html">http://www.campin.net/syslog-ng/faq.html</a>
</pre>
</blockquote>
</body>
</html>