[syslog-ng] Parsing Question

Balazs Scheidler bazsi at balabit.hu
Mon Aug 1 20:23:28 CEST 2011


On Fri, 2011-07-29 at 19:22 +0200, Jakub Jankowski wrote:
> On 2011-07-29, Brandon Phelps wrote:
> 
> > Could anyone explain how I would parse a message that looks like this:
> > Jul 29 08:58:38 192.168.1.1 id=firewall sn=0017C5158708 time="2011-07-29
> > 08:58:38" fw=100.1.1.1 pri=6 c=262144 m=98 msg="Connection Opened" n=0
> > src=192.168.2.100:123:X0 dst=74.1.2.3:X1 proto=udp/ntp
> >
> > I am logging to mysql and would like to extract the 'src' and 'dst'
> > fields from the above message so that I can insert them into indexed
> > fields in my database.
> [...]
> > Is my only option in this case to write a perl script or something that
> > watches a named pipe and have syslog-ng log to the named pipe instead,
> > while my perl script does the actual parsing?  Or can I do what I want
> > with syslog-ng alone?
> 
> You seriously need to look at patterndb functionality.
> http://bazsi.blogs.balabit.com/2009/03/an-introduction-to-db-parser/
> http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.2-guide-admin-en.html/index.html-single.html#chapter-patterndb

patterndb() would work if the order of the fields is definite. if they
are not, it's going to be ugly. I was pondering to write a welf parser
(which the above format is), that could  be used to preprocess logs
prior to going to db-parser(), but that's something that you either have
to wait for, implement yourself or wait someone who has the same itch
and does it for you. :)

-- 
Bazsi




More information about the syslog-ng mailing list