[syslog-ng] Cant get pattern matching to work

[Chris Johnson]

Even though the documentation shows that you need to escape a colon as a stop character, you don't need to. You are exactly right. Works!

Thanks for your help!!!!!

On Feb 11, 2012, at 6:45 PM, Evan Rempel wrote:

> It also looks like you have an extra colen in your ESTRING.
> 
> The format should be
> 
> @ESTRING:varname:stop_character@
> ________________________________________
> From: syslog-ng-bounces at lists.balabit.hu [syslog-ng-bounces at lists.balabit.hu] On Behalf Of Evan Rempel [erempel at uvic.ca]
> Sent: Saturday, February 11, 2012 6:28 PM
> To: Syslog-ng users' and developers' mailing list
> Subject: Re: [syslog-ng] Cant get pattern matching to work
> 
> It may be because you are trying to ESTRING with a colen (:)
> 
> I don't have the docs in front of me, but there should be some special syntax to ESTRING with a colen (:).
> 
> ________________________________________
> From: syslog-ng-bounces at lists.balabit.hu [syslog-ng-bounces at lists.balabit.hu] On Behalf Of Chris Johnson [masterof0 at gmail.com]
> Sent: Saturday, February 11, 2012 6:16 PM
> To: Syslog-ng users' and developers' mailing list
> Subject: Re: [syslog-ng] Cant get pattern matching to work
> 
> Thanks for the feedback. One thing I noticed is that your pattern definition is inside the ruleset which was an initial error on my part. Now that is fixed and I still had the same challenge. Based on some trouble shooting, I have narrowed it down to the ESTRING definition. No matter what I try with the ESTRING definition, I cannot get it to work. If I do an exact pattern match, such as RT_FLOW_SESSION_CLOSE and remove the ESTRING defntion, everything works as expected.
> 
> Thoughts?
> 
> Chris
> 
> On Feb 11, 2012, at 5:12 PM, Evan Rempel wrote:
> 
>> you need to have a pattern for your ruleset, which will match the $program macro.
>> 
>> <patterndb version='3' pub_date='2011-02-11'>
>>   <ruleset name='session_close' id='123456678'>
>>       <pattern>RT_FLOW</pattern>
>>       <rules>
>>           <rule provider='cj' id='182437592347598' class='session'>
>>           <patterns>
>>               <pattern>@ESTRING:TEST1:::@ @ANYSTRING:TEST2@</pattern>
>>               </patterns>
>>           </rule>
>>       </rules>
>>   </ruleset>
>> </patterndb>
>> 
>> Then TEST1 should be
>> RT_FLOW_SESSION_CLOSE
>> 
>> and TEST2 should be
>> session closed unset: 192.168.199.253/55189->8.8.8.8/53 junos-dns-udp X.X.X.X/60836->8.8.8.8/53 source-nat-rule None 17 trust-to-untrust trust untrust 30455 1(83) 1(83) 2 N/A N/A N/A(N/A) vlan.0
>> 
>> home that helps.
>> 
>> Evan.
>> ________________________________________
>> From: syslog-ng-bounces at lists.balabit.hu [syslog-ng-bounces at lists.balabit.hu] On Behalf Of Chris Johnson [masterof0 at gmail.com]
>> Sent: Saturday, February 11, 2012 4:44 PM
>> To: Syslog-ng and developers' mailing list users'
>> Subject: [syslog-ng] Cant get pattern matching to work
>> 
>> Sorry for bombarding the mailing list with questions. I have been working through the documentation but just seem to keep hitting obstacles. I have compared this to the examples in the manual and some samples I was able to track down, but Im not seeing where I am going wrong. For this question, I am working with the following data:
>> 
>> "MESSAGE" : "RT_FLOW_SESSION_CLOSE: session closed unset: 192.168.199.253/55189->8.8.8.8/53 junos-dns-udp X.X.X.X/60836->8.8.8.8/53 source-nat-rule None 17 trust-to-untrust trust untrust 30455 1(83) 1(83) 2 N/A N/A N/A(N/A) vlan.0",
>> "PROGRAM" : "RT_FLOW"
>> 
>> Based on this , I have generated the following pattern database and config just to see if I am heading down the right path. Unfortunately, what I thought would generate a pattern match on the logs and generate the appropriate macros, are not retuning anything. I have also tried taking out the $PROGRAM patter, but that did not seem to help. I have also validated this with pdbtool and it succeeds.
>> 
>> Any help is greatly appreciated
>> Chris
>> 
>> patterndb.xml
>> 
>> <patterndb version='3' pub_date='2011-02-11'>
>> <ruleset name='session_close' id='123456678'>
>> <rules>
>> <rule provider='cj' id='182437592347598' class='session'>
>> <patterns>
>> <pattern>
>> @ESTRING:TEST1:::@ @ANYSTRING:TEST2@
>> </pattern>
>> </patterns>
>> </rule>
>> </rules>
>> </ruleset>
>> </patterndb>
>> 
>> 
>> syslog-ng.conf
>> 
>> #####Destinations#####
>> destination d_mongodb {
>>       mongodb(
>>               value-pairs(
>>                       key("TEST1")
>>                       key("TEST2")
>>                       scope("base")
>>               )
>>       );
>> };
>> 
>> #####Parser#####
>> parser pattern_db {
>>       db_parser(
>>               file("/usr/local/etc/patterndb.xml")
>>       );
>> };
>> 
>> #####Log#####
>> log {
>>       source(s_network);
>>       parser(pattern_db);
>>       destination(d_mongodb);
>> };
>> 
>> 
>> Unfortunately, what I thought would pattern match and generate a macro for
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>> 
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>