[syslog-ng] log failback groups
Patrick H.
syslogng at feystorm.net
Mon Mar 22 15:34:24 CET 2010
Ok, so finally got time to look at this again.
I tried the time_sleep option, and it was a horrible failure. With it
enabled syslog-ng started losing about 98% of all incoming lines (and
no, thats not an exaggeration). I'm guessing that time_sleep does not
play well with udp as thats what incoming data is being sent over.
However I do have the master-slave multi-process thing going and its
working really good. I was able to put time_sleep on the child processes
(the one doing regex matches), and it dropped their cpu utilization from
around 40% to about 20% (master process uses tcp to talk to slave
processes, so no drops). Another thing is that when I tried using the
syslog protocol to talk to the child processes, the slaves were
terminating the connection within seconds of being established. I poked
and prodded and could not get this to work without constantly dropping
the connection, so I had to switch back to plain tcp.
Anyway, the attached config is what it looked like when I had all
regexes run within a single process (the config that was utilizing over
90% cpu).
Sent: Thursday, March 18, 2010 10:56:16 PM
From: Jan Schaumann <jschauma at netmeister.org>
To: syslog-ng at lists.balabit.hu
Subject: Re: [syslog-ng] log failback groups
> Martin Holste <mcholste at gmail.com> wrote:
>
>> How many messages per second is the system attempting to handle? I'm
>> very surprised that you're seeing that level of utilization. In our
>> setup we've never had a problem pushing up through 30,000 messages per
>> second written to disk with Syslog-NG in production, and I've pushed
>> more than 70,000 per second in development.
>>
>
> Could you provide your configuration for these systems (including
> sysctls or kernel tunables etc.)? I've so far not been able to get my
> systems to accept and process (without any regex matching) more than
> approximagely 25K - 30K UDP messages/s.
>
> -Jan
>
> ------------------------------------------------------------------------
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100322/109aad6f/attachment.htm
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: syslog-ng.conf
Url: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100322/109aad6f/attachment.txt
More information about the syslog-ng
mailing list