[syslog-ng] Separating Remote Logs
Nate Hausrath
hausrath.mailing.list at gmail.com
Mon Jan 11 15:55:27 CET 2010
Thanks again for your response Martin. Unfortunately, it is still not
parsing correctly and I am not sure where to go from here. I can try
switching to the pattern db 3.0, but where do I find information about
the differences?
Right now, the ASA logs are being placed in the other.log file, and no
other logs are being placed anywhere (even though I have verified they
are being received). Just to reiterate, I'm trying to place the
Windows logs in a windows.log file, ASA logs in an asa.log file, and
everything else in the other.log file.
Also, just in case, here is my latest XML file:
<?xml version='1.0' encoding='UTF-8'?>
<patterndb version='2' pub_date='2009-12-07'>
<ruleset name='asa' id='1'>
<pattern>%ASA</pattern>
<rules>
<rule provider='capc' id='1' class='system'>
<description>Detects ASA logs</description>
<patterns>
<pattern></pattern>
</patterns>
</rule>
</rules>
</ruleset>
<ruleset name='win' id='2'>
<pattern>MSWinEventLog</pattern>
<rules>
<rule provider='capc' id='2' class='system'>
<description>Detects Windows logs from Snare</description>
<patterns>
<pattern></pattern>
</patterns>
</rule>
</rules>
</ruleset>
</patterndb>
And here are the relevant parts of my syslog-ng.conf file:
destination df_asa { file("/var/log/remote/asa.log"); };
destination df_windows { file("/var/log/remote/windows.log"); };
destination df_other { file("/var/log/remote/other.log"); };
filter f_class_asa {
match("1"
value(".classifier.rule_id")
type("string")
);
};
filter f_class_windows {
match("2"
value(".classifier.rule_id")
type("string")
);
};
filter f_class_other {
not match("1" value(".classifier.rule_id") type("string"))
and not match("2" value(".classifier.rule_id") type("string"));
};
parser p_capc{
db-parser( file("/opt/syslog-ng/var/capcdb2.xml") );
};
log {
source(s_remote);
parser(p_capc);
filter(f_class_asa);
destination(df_asa);
};
log {
source(s_remote);
parser(p_capc);
filter(f_class_windows);
destination(df_windows);
};
log {
source(s_remote);
parser(p_capc);
filter(f_class_other);
destination(df_other);
};
It should be relatively straightforward, so I must be overlooking something.
I've tested things with pdbtool as well, but it's not giving me the
responses I would expect.
# pdbtool dump -p /opt/syslog-ng/var/capcdb2.xml -P '%ASA'
''
'' rule_id='1'
# pdbtool match -p /opt/syslog-ng/var/capcdb2.xml -P "%ASA" -M "Jan 6
13:49:22 10.X.X.X %ASA-5-304001: 10.Y.Y.Y Accessed URL "
MESSAGE=Jan 6 13:49:22 10.48.8.111 %ASA-5-304001: 10.48.10.82 Accessed URL
PROGRAM=%ASA
.classifier.class=unknown
Thanks again for any suggestions.
-Nate
On Fri, Jan 8, 2010 at 3:01 PM, Martin Holste <mcholste at gmail.com> wrote:
>
> The pattern is a bit misleading in the 2.0 pattern db schema, as it
> means the pattern of the $PROGRAM macro sometimes, and the $MSG macro
> within a rule element. I think what you want is this:
>
> <?xml version='1.0' encoding='UTF-8'?>
> <patterndb version='2' pub_date='2009-12-07'>
> <ruleset name='capcxml' id='1923-ab2b'>
> <pattern>%ASA</pattern>
> <rules>
> <rule provider='capc' id='1' class='system'>
> <description>Detects ASA logs</description>
> <patterns>
> <pattern></pattern>
> </patterns>
> </rule>
> </rules>
> </ruleset>
> <ruleset>
> <pattern>MSWinEventLog</pattern>
> <rules>
> <rule provider='capc' id='2' class='system'>
> <description>Detects Windows logs from Snare</description>
> <patterns>
> <pattern></pattern>
> </patterns>
> </rule>
> </rules>
> </patterndb>
>
> You want to consider switching over to the 3.0 patterndb version as it
> is a bitmore user-friendly in my opinion. There are 2.0 examples
> here: http://www.balabit.com/downloads/files/patterndb/1.0-20081117/patterndb/patterndb.xml
> .
>
> --Martin
>
> On Fri, Jan 8, 2010 at 10:45 AM, Nate Hausrath
> <hausrath.mailing.list at gmail.com> wrote:
> > Thanks for your response.
> >
> > I must be doing something else wrong, because after changing everything to .classifier.rule_id, all my logs are placed in the other.log file. Is there anything else that jumps out at you?
> >
> > Thanks,
> > Nate
> >
> > On Jan 7, 2010, at 1:27 PM, Martin Holste wrote:
> >
> >> It should be ".classifier.rule_id" not ".classifier_rule_id."
> >>
> >> By the way, I find I need to use a rewrite template for Snare to
> >> normalize the program. I use:
> >>
> >> rewrite r_snare {
> >> subst("MSWinEventLog.+(Security|Application|System).+", "$1",
> >> value("PROGRAM") flags(global));
> >> };
> >>
> >> --Martin
> >>
> >> On Thu, Jan 7, 2010 at 11:56 AM, Nate Hausrath
> >> <hausrath.mailing.list at gmail.com> wrote:
> >>> Hi everyone,
> >>>
> >>> I've set up a central log server using syslog-ng that receives logs from many different locations on the network. I want to break these logs up into different files so I can using another program to parse and interpret each one individually. So for example, I want my Windows logs to go to /var/log/remote/windows.log, which my ASA logs go to /var/log/remote/asa.log. Anything that doesn't fit into a group should go to /var/log/remote/other.log.
> >>>
> >>> I'm trying to figure out the best way to do this. I was going to create a custom xml file for the db-parser, but I can't seem to get it to work the way I want. Here is my db-parser XML file so far:
> >>>
> >>> <?xml version='1.0' encoding='UTF-8'?>
> >>> <patterndb version='2' pub_date='2009-12-07'>
> >>> <ruleset name='capcxml' id='1923-ab2b'>
> >>> <pattern/>
> >>> <rules>
> >>> <rule provider='capc' id='1' class='system'>
> >>> <description>Detects ASA logs</description>
> >>> <patterns>
> >>> <pattern>%ASA</pattern>
> >>> </patterns>
> >>> </rule>
> >>> <rule provider='capc' id='2' class='system'>
> >>> <description>Detects Windows logs from Snare</description>
> >>> <patterns>
> >>> <pattern>MSWinEventLog</pattern>
> >>> </patterns>
> >>> </rule>
> >>> </rules>
> >>> </ruleset>
> >>> </patterndb>
> >>>
> >>> So my question is, how can I specify a filter in my syslog-ng.conf file that will allow me to separate logs from these two different rules? Along with that, how can I separate everything that doesn't match?
> >>>
> >>> For instance, I could have something like this:
> >>>
> >>> filter f_class_asa {
> >>> match("1"
> >>> value(".classifier_rule_id")
> >>> type("string")
> >>> );
> >>> };
> >>>
> >>> filter f_class_windows {
> >>> match("2"
> >>> value(".classifier_rule_id")
> >>> type("string")
> >>> );
> >>> };
> >>>
> >>> filter f_class_other {
> >>> not match("1" value(".classifier_rule_id") type("string"))
> >>> and not match("2" value(".classifier_rule_id") type("string"));
> >>> };
> >>>
> >>> log {
> >>> source(s_remote);
> >>> parser(p_capc);
> >>> filter(f_class_asa);
> >>> destination(df_asa);
> >>> };
> >>>
> >>> log {
> >>> source(s_remote);
> >>> parser(p_capc);
> >>> filter(f_class_windows);
> >>> destination(df_windows);
> >>> };
> >>>
> >>> log {
> >>> source(s_remote);
> >>> parser(p_capc);
> >>> filter(f_class_other);
> >>> destination(df_other);
> >>> };
> >>>
> >>> But this doesn't seem to work. Everything gets placed in the df_asa file.
> >>>
> >>> Am I approaching this the wrong way? Should I not use db-parser for this task?
> >>>
> >>> Thanks for any help!
> >>> Nate
> >>> ______________________________________________________________________________
> >>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> >>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> >>> FAQ: http://www.campin.net/syslog-ng/faq.html
> >>>
> >>>
> >> ______________________________________________________________________________
> >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> >> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> >> FAQ: http://www.campin.net/syslog-ng/faq.html
> >>
> >
> > ______________________________________________________________________________
> > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> > FAQ: http://www.campin.net/syslog-ng/faq.html
> >
> >
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
More information about the syslog-ng
mailing list