[syslog-ng] JSON vs PatternDB for parser performance

[Balazs Scheidler]

On Tue, 2012-01-17 at 11:28 -0600, Martin Holste wrote:
> Does anyone have a good idea as to which parser is fastest?  The
> pattern matching tree of PatternDB makes me think that it will be
> faster, but I'd like to see what others think.  Testing in my
> environment would be non-trivial, so I am hoping someone already has.

I don't have hard data, but I believe that patterndb should be faster as
long as single threaded performance is to be judged. 

json-parser() should scale linearly for the number of connections
(modulo worker threads), since it is not using any kind of shared state,
whereas db-parser() uses only a single copy of the radix tree and as of
now, a reader/writer lock is taken before trying to match a rule.

json-parser() seems to use a hand-coded lexer/grammar, but does seem to
allocate memory for all data it stores, so even if the parser is good
enough, memory allocation would probably dominate the CPU load.

-- 
Bazsi