[syslog-ng] log failback groups

Patrick H. syslogng at feystorm.net
Mon Mar 22 15:34:24 CET 2010


Ok, so finally got time to look at this again.
I tried the time_sleep option, and it was a horrible failure. With it 
enabled syslog-ng started losing about 98% of all incoming lines (and 
no, thats not an exaggeration). I'm guessing that time_sleep does not 
play well with udp as thats what incoming data is being sent over.
However I do have the master-slave multi-process thing going and its 
working really good. I was able to put time_sleep on the child processes 
(the one doing regex matches), and it dropped their cpu utilization from 
around 40% to about 20% (master process uses tcp to talk to slave 
processes, so no drops). Another thing is that when I tried using the 
syslog protocol to talk to the child processes, the slaves were 
terminating the connection within seconds of being established. I poked 
and prodded and could not get this to work without constantly dropping 
the connection, so I had to switch back to plain tcp.

Anyway, the attached config is what it looked like when I had all 
regexes run within a single process (the config that was utilizing over 
90% cpu).



Sent: Thursday, March 18, 2010 10:56:16 PM
From: Jan Schaumann <jschauma at netmeister.org>
To: syslog-ng at lists.balabit.hu
Subject: Re: [syslog-ng] log failback groups
> Martin Holste <mcholste at gmail.com> wrote:
>   
>> How many messages per second is the system attempting to handle?  I'm
>> very surprised that you're seeing that level of utilization.  In our
>> setup we've never had a problem pushing up through 30,000 messages per
>> second written to disk with Syslog-NG in production, and I've pushed
>> more than 70,000 per second in development.
>>     
>
> Could you provide your configuration for these systems (including
> sysctls or kernel tunables etc.)?  I've so far not been able to get my
> systems to accept and process (without any regex matching) more than
> approximagely 25K - 30K UDP messages/s.
>  
> -Jan
>   
> ------------------------------------------------------------------------
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100322/109aad6f/attachment.htm 
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: syslog-ng.conf
Url: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100322/109aad6f/attachment.txt 


More information about the syslog-ng mailing list