[syslog-ng] Syslog-NG to parse HTTP query strings?

[Balazs Scheidler]

On Fri, 2010-07-09 at 19:16 +1000, Steve Barnes wrote:
> > On Fri, 2010-07-09 at 16:51 +1000, Steve Barnes wrote:
> >> Hello
> >>
> >> Is Syslog-NG capable of parsing HTTP query string key=value pairs? I've
> >> read through both the CSV and pattern DB parser documentation and it
> >> seems
> >> they both presume a consistent ordering in the $MSG field. Neither seem
> >> appropriate when if the order of query string pairs changes, such as:
> >>
> >> http://somehost.com/somefile.cgi?keyA=valueA&keyB=valueB
> >>
> >> or
> >>
> >> http://somehost.com/somefile.cgi?keyB=valueB&keyA=valueA
> >>
> >> Basically, I want to grab keyA's value irrespective of which order it
> >> appears and use it as a value in an SQL insert statement. Is this
> >> possible
> >> with Syslog-NG's native parsers or am I going to have to handle this in
> >> a
> >> Perl script/program?
> >
> > Not right now, however syslog-ng 3.2 (only in its git form right now),
> > features plugins and it is quite easy to implement a new parser using  a
> > plugin.
> >
> > using the internal API of syslog-ng is _way_ faster than using an
> > external script.
> 
> Bazsi
> 
> Agreed - handling parsing within Syslog-NG is definitely my preferred
> choice. I'll take a look at 3.2 and see what's involved in writing a
> plugin. Thanks for the pointer.

Let me know if I can give you help. I've now split the db-parser() and
csv-parser() code into separate plugins.

The tricky part is the configuration parser, since bison doesn't support
modular parsers, but if you copy the makefiles, *-parser.{c,h} files of
the other modules and modify them accordingly should be relatively
straightforward.

-- 
Bazsi