[syslog-ng] JSON vs PatternDB for parser performance

[Balazs Scheidler]

On Thu, 2012-01-19 at 16:48 -0600, Martin Holste wrote:
> Ok, thanks.  And for good measure, I assume that the CSV parser is the fastest?

should be, yes.

> 
> On Thu, Jan 19, 2012 at 8:59 AM, Balazs Scheidler <bazsi at balabit.hu> wrote:
> > On Tue, 2012-01-17 at 11:28 -0600, Martin Holste wrote:
> >> Does anyone have a good idea as to which parser is fastest?  The
> >> pattern matching tree of PatternDB makes me think that it will be
> >> faster, but I'd like to see what others think.  Testing in my
> >> environment would be non-trivial, so I am hoping someone already has.
> >
> > I don't have hard data, but I believe that patterndb should be faster as
> > long as single threaded performance is to be judged.
> >
> > json-parser() should scale linearly for the number of connections
> > (modulo worker threads), since it is not using any kind of shared state,
> > whereas db-parser() uses only a single copy of the radix tree and as of
> > now, a reader/writer lock is taken before trying to match a rule.
> >
> > json-parser() seems to use a hand-coded lexer/grammar, but does seem to
> > allocate memory for all data it stores, so even if the parser is good
> > enough, memory allocation would probably dominate the CPU load.
> >
> > --
> > Bazsi
> >
> >
> > ______________________________________________________________________________
> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> > FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> >
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> 
> 

-- 
Bazsi