On Fri, 2011-07-29 at 19:22 +0200, Jakub Jankowski wrote:
On 2011-07-29, Brandon Phelps wrote:
Could anyone explain how I would parse a message that looks like this: Jul 29 08:58:38 192.168.1.1 id=firewall sn=0017C5158708 time="2011-07-29 08:58:38" fw=100.1.1.1 pri=6 c=262144 m=98 msg="Connection Opened" n=0 src=192.168.2.100:123:X0 dst=74.1.2.3:X1 proto=udp/ntp
I am logging to mysql and would like to extract the 'src' and 'dst' fields from the above message so that I can insert them into indexed fields in my database. [...] Is my only option in this case to write a perl script or something that watches a named pipe and have syslog-ng log to the named pipe instead, while my perl script does the actual parsing? Or can I do what I want with syslog-ng alone?
You seriously need to look at patterndb functionality. http://bazsi.blogs.balabit.com/2009/03/an-introduction-to-db-parser/ http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.2-guid...
patterndb() would work if the order of the fields is definite. if they are not, it's going to be ugly. I was pondering to write a welf parser (which the above format is), that could be used to preprocess logs prior to going to db-parser(), but that's something that you either have to wait for, implement yourself or wait someone who has the same itch and does it for you. :) -- Bazsi