[syslog-ng] Cant get pattern matching to work

[Evan Rempel]

It also looks like you have an extra colen in your ESTRING.

The format should be

@ESTRING:varname:stop_character@
________________________________________
From: syslog-ng-bounces at lists.balabit.hu [syslog-ng-bounces at lists.balabit.hu] On Behalf Of Evan Rempel [erempel at uvic.ca]
Sent: Saturday, February 11, 2012 6:28 PM
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] Cant get pattern matching to work

It may be because you are trying to ESTRING with a colen (:)

I don't have the docs in front of me, but there should be some special syntax to ESTRING with a colen (:).

________________________________________
From: syslog-ng-bounces at lists.balabit.hu [syslog-ng-bounces at lists.balabit.hu] On Behalf Of Chris Johnson [masterof0 at gmail.com]
Sent: Saturday, February 11, 2012 6:16 PM
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] Cant get pattern matching to work

Thanks for the feedback. One thing I noticed is that your pattern definition is inside the ruleset which was an initial error on my part. Now that is fixed and I still had the same challenge. Based on some trouble shooting, I have narrowed it down to the ESTRING definition. No matter what I try with the ESTRING definition, I cannot get it to work. If I do an exact pattern match, such as RT_FLOW_SESSION_CLOSE and remove the ESTRING defntion, everything works as expected.

Thoughts?

Chris

On Feb 11, 2012, at 5:12 PM, Evan Rempel wrote:

> you need to have a pattern for your ruleset, which will match the $program macro.
>
> <patterndb version='3' pub_date='2011-02-11'>
>    <ruleset name='session_close' id='123456678'>
>        <pattern>RT_FLOW</pattern>
>        <rules>
>            <rule provider='cj' id='182437592347598' class='session'>
>            <patterns>
>                <pattern>@ESTRING:TEST1:::@ @ANYSTRING:TEST2@</pattern>
>                </patterns>
>            </rule>
>        </rules>
>    </ruleset>
> </patterndb>
>
> Then TEST1 should be
> RT_FLOW_SESSION_CLOSE
>
> and TEST2 should be
> session closed unset: 192.168.199.253/55189->8.8.8.8/53 junos-dns-udp X.X.X.X/60836->8.8.8.8/53 source-nat-rule None 17 trust-to-untrust trust untrust 30455 1(83) 1(83) 2 N/A N/A N/A(N/A) vlan.0
>
> home that helps.
>
> Evan.
> ________________________________________
> From: syslog-ng-bounces at lists.balabit.hu [syslog-ng-bounces at lists.balabit.hu] On Behalf Of Chris Johnson [masterof0 at gmail.com]
> Sent: Saturday, February 11, 2012 4:44 PM
> To: Syslog-ng and developers' mailing list users'
> Subject: [syslog-ng] Cant get pattern matching to work
>
> Sorry for bombarding the mailing list with questions. I have been working through the documentation but just seem to keep hitting obstacles. I have compared this to the examples in the manual and some samples I was able to track down, but Im not seeing where I am going wrong. For this question, I am working with the following data:
>
> "MESSAGE" : "RT_FLOW_SESSION_CLOSE: session closed unset: 192.168.199.253/55189->8.8.8.8/53 junos-dns-udp X.X.X.X/60836->8.8.8.8/53 source-nat-rule None 17 trust-to-untrust trust untrust 30455 1(83) 1(83) 2 N/A N/A N/A(N/A) vlan.0",
> "PROGRAM" : "RT_FLOW"
>
> Based on this , I have generated the following pattern database and config just to see if I am heading down the right path. Unfortunately, what I thought would generate a pattern match on the logs and generate the appropriate macros, are not retuning anything. I have also tried taking out the $PROGRAM patter, but that did not seem to help. I have also validated this with pdbtool and it succeeds.
>
> Any help is greatly appreciated
> Chris
>
> patterndb.xml
>
> <patterndb version='3' pub_date='2011-02-11'>
> <ruleset name='session_close' id='123456678'>
> <rules>
> <rule provider='cj' id='182437592347598' class='session'>
> <patterns>
> <pattern>
> @ESTRING:TEST1:::@ @ANYSTRING:TEST2@
> </pattern>
> </patterns>
> </rule>
> </rules>
> </ruleset>
> </patterndb>
>
>
> syslog-ng.conf
>
> #####Destinations#####
> destination d_mongodb {
>        mongodb(
>                value-pairs(
>                        key("TEST1")
>                        key("TEST2")
>                        scope("base")
>                )
>        );
> };
>
> #####Parser#####
> parser pattern_db {
>        db_parser(
>                file("/usr/local/etc/patterndb.xml")
>        );
> };
>
> #####Log#####
> log {
>        source(s_network);
>        parser(pattern_db);
>        destination(d_mongodb);
> };
>
>
> Unfortunately, what I thought would pattern match and generate a macro for
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq