On Thu, 2009-05-07 at 16:26 -0400, Nate Hausrath wrote:
Hey everyone,
Sorry to ask what is probably an easy question.
I'm attempting to use the new db-parser feature in syslog-ng to parse Windows events generated by Snare. What I've attempted to do is write a simple XML file that will match one simple rule. This way I can test that everything is functioning properly and I can move on from there. Unfortunately, it's not working, so I'm unsure as to whether it is my rule or my installation. So I have a few basic questions that will hopefully clear up my confusion!
- I did not install syslog-ng from source. I downloaded the .deb for version 3.01 and used that. Does this even include db-parser? (I'm assuming yes because it does not cause an error from the config file alone.)
- My installation is in /opt/syslog-ng, so I've put the example XML files (http://www.balabit.com/downloads/files/patterndb/) in /opt/syslog-ng/var. Is this the right location?
the default location for the patterndb file is ${localstatedir}/patterndb.xml, but you can override that using the file option, e.g. db-parser(file(/path/to/patterndb)); the reason it is in var that in the future we assume that this file is going to be generated from several source files automatically.
Other than that, here is my syslog-ng.conf file:
@version: 3.0 #Default configuration file for syslog-ng. # # For a description of syslog-ng configuration file directives, please read # the syslog-ng Administrator's guide at: # # http://www.balabit.com/dl/html/syslog-ng-admin-guide_en.html/bk01-toc.html #
options { };
###### # sources source s_local { # message generated by Syslog-NG internal(); # standard Linux log source (this is the default place for the syslog() # function to send logs to) unix-stream("/dev/log"); # messages from the kernel file("/proc/kmsg" program_override("kernel: ")); };
source s_net { udp(); tcp(); };
parser p_db { db-parser(); };
###### # destinations destination d_messages { file("/var/log/messages"); }; destination d_parsed { file("/var/log/remote/parsed.log"); };
log { source(s_local); destination(d_messages); };
log { source(s_net); destination(d_parsed); parser(p_db); };
the log statements define a 'pipeline', thus your messages will reach the db-parser() only after having been written to the d_parsed destination.
And here is my windows.xml file:
<?xml version='1.0' encoding='UTF-8'?> <patterndb version='2' pub_date='2009-05-07'> <ruleset name='windows'> <pattern>MSWinEventLog</pattern> <rules> <rule provider='nate' id='1' class='system'> <patterns> <pattern>540</pattern> </patterns> <description>This is a terrible terrible message to receive. Game over man! Game over!</description> </rule> </rules> </ruleset> </patterndb>
As you see, to test I just want to match on the number '540' and put that log message in /var/log/remote/parsed.log. I have verified that messages that should match this are arriving at the machine when I send them. If i turn off the db-parser, the messages appear in the parsed.log file.
All messages should end up in your parsed.log file anyway. In order to only match those which matched any of the patterns, you need to put filters into your log statement.
If there is any documentation that explains the basics of this, sorry that I missed it. I've only been able to find some of the other messages on this mailing list and the blog entries on balabit.
Just feel free to ask about it here, I'm trying to give you timely answers, as other source of information about db-parser() is scarse. In the documentation you will find information about parsers in general, and db-parser() is one such parser, basically behaving quite similar to csv-parser(), except it uses a database to extract information from log messages. Also, you can find a working example in my presentation at OSDC: http://people.balabit.hu/bazsi/slides/osdc-2009-syslog-ng-3.0.odp -- Bazsi