[syslog-ng] log failback groups

[Martin Holste]

That's definitely some heavy-duty regex.  You'd be a good candidate for the
pattern db, as the pattern matching engine is orders of magnitude faster
than PCRE because it uses trie-based pattern searching.  It also allows for
extracting the matches and using them in the output macros, so you wouldn't
have to sacrifice any functionality.  I would estimate that it would drop
your CPU usage down to around 25-30% while doing all of the work in a single
thread.

@Balabit: You know what goes great with pattern matching?  CUDA support with
Nvidia cards for GPU-based pattern matching acceleration.  They've got
preliminary support for it in the Open Information Security Foundation's
(OISF) Suricata IDS engine.  That project is GPL, so you could port most of
that code directly into the pattern db matcher for the OSE version of
SyslogNG.  $500 USD will buy a GPU with 480 stream processors, so you could
match 480 patterns simultaneously, per card.  You can link up to four cards
together, so you could match 1920 patterns in parallel, offloaded from the
CPU, on commodity hardware.  So, a server costing under $5,000 could
probably process (maybe not store) 250,000+ messages per second.  Even if
there wasn't much speed increase, the CPU offload alone would probably be
worth it for busy log servers.

On Mon, Mar 22, 2010 at 11:50 AM, Balazs Scheidler <bazsi at balabit.hu> wrote:

> On Fri, 2010-03-19 at 00:56 -0400, Jan Schaumann wrote:
> > Martin Holste <mcholste at gmail.com> wrote:
> > > How many messages per second is the system attempting to handle?  I'm
> > > very surprised that you're seeing that level of utilization.  In our
> > > setup we've never had a problem pushing up through 30,000 messages per
> > > second written to disk with Syslog-NG in production, and I've pushed
> > > more than 70,000 per second in development.
> >
> > Could you provide your configuration for these systems (including
> > sysctls or kernel tunables etc.)?  I've so far not been able to get my
> > systems to accept and process (without any regex matching) more than
> > approximagely 25K - 30K UDP messages/s.
> >
>
> udp() may be the problem here, in fact since syslog-ng is not thread
> based, its latency to poll for the udp socket may be a bit too long.
>
> increasing the udp socket buffer to insane values (like 256MB) could
> help alleviate the problem somewhat, but the latency is the root cause.
>
> adding time_sleep() to the mix increases latency even further. It only
> helps if you have a lot of connections (and udp is only one even if you
> have thousands of clients, while tcp keeps a separate connection for
> each client).
>
> After I get to the end of integrating the plugins branch, I intend to
> work on enabling multiple threads, thus decreasing the latency.
>
>
> --
> Bazsi
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100322/06e4524b/attachment.htm