[syslog-ng] JSON vs PatternDB for parser performance

[Martin Holste]

Ok, thanks.  And for good measure, I assume that the CSV parser is the fastest?

On Thu, Jan 19, 2012 at 8:59 AM, Balazs Scheidler <bazsi at balabit.hu> wrote:
> On Tue, 2012-01-17 at 11:28 -0600, Martin Holste wrote:
>> Does anyone have a good idea as to which parser is fastest?  The
>> pattern matching tree of PatternDB makes me think that it will be
>> faster, but I'd like to see what others think.  Testing in my
>> environment would be non-trivial, so I am hoping someone already has.
>
> I don't have hard data, but I believe that patterndb should be faster as
> long as single threaded performance is to be judged.
>
> json-parser() should scale linearly for the number of connections
> (modulo worker threads), since it is not using any kind of shared state,
> whereas db-parser() uses only a single copy of the radix tree and as of
> now, a reader/writer lock is taken before trying to match a rule.
>
> json-parser() seems to use a hand-coded lexer/grammar, but does seem to
> allocate memory for all data it stores, so even if the parser is good
> enough, memory allocation would probably dominate the CPU load.
>
> --
> Bazsi
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>