Re: [syslog-ng] Problems with custom patterndb

29 Nov 2011

      Thanks for the help.

I checked over the configuration, top-to-bottom, and there's no sign of the "no_parse" flag.  Still no luck though.  I read in the admin guide about triggered messages having to use the inject_mode(internal), I'm not sure if this applies here - but I'm not specifying anything.  My ultimate goal is correlate events together into a single message which will go to program().  As of now though, I can't even get patterndb messages to get logged to flat files. 

Here's more detail:

Syslog-NG version 3.3
Using loggen to generate the logs with an input file as the source of the messages:
  /opt/syslog-ng/bin/loggen -i -D -R ~fclements/syslog.messages  --skip-tokens=1 192.168.1.10 515

The input file contains:
 NAT-UDP-C: 192.168.107.132:12260 -> 207.44.101.104:12260 to 48.70.67.223:940
 UC: c0a80b1b:1a->c0a8650b:5c40 to c0a8650c:3f

I have the following patterns created (from patterndb.xml):
         <rule id='18795aa7-eb96-454d-b954-646c0b679d44' class='system' provider='testing-provider'>
            <patterns>
               <pattern> 
                  @IPvANY:.dict.insideAddr@:@NUMBER:.dict.insidePort@ -> @IPvANY:.dict.outsideAddr@:@NUMBER:.dict.outsidePort@ to @IPvANY:.dict.destAddr@:@NUMBER:.dict.destPort@ 
               </pattern>
               <pattern>
                  @STRING:.dict.insideAddr@:@STRING:.dict.insidePort@->@STRING:.dict.outsideAddr@:@ESTRING:.dict.outsidePort: @to @STRING:.dict.destAddr@:@ESTRING:.dict.destPort: @
               </pattern>
            </patterns>
            <values></values>
         </rule>

syslog-ng.conf has the following:
options {
   threaded(yes);
   use_dns(no);
};

parser pattern_db {
   db_parser( file("/var/lib/syslog-ng/patterndb.xml") );
};

source s_remote {
	udp( ip(0.0.0.0) port(515) flags("store-legacy-msghdr"));
};
destination d_message_class {
	file( "/var/log/parsed/messages"
           template( "${.classifier.rule_id}\t${S_UNIXTIME}\t${HOST}\t${PROGRAM}\t${.dict.insideAddr}\t${.dict.insidePort}\n" )
           create_dirs(yes)
	);
}; 

log {
	source(s_remote);
	parser(pattern_db);
	destination( d_message_class );
};

In my template I'm using .classifier.rule_id, .dict.insideAddr and .dict.insidePort.  They all turn up blank in the file - I'm not even getting a hit on the rule_id:
	1322539328	192.168.1.10	NAT-UDP-C		
	1322539328	192.168.1.10	UC		

-Frank

________________________________________
From: syslog-ng-bounces@lists.balabit.hu [syslog-ng-bounces@lists.balabit.hu] On Behalf Of Evan Rempel [erempel@uvic.ca]
Sent: Monday, November 28, 2011 5:28 PM
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] Problems with custom patterndb

What you have is correct (I tested it on my parser database)
given that the message getting passed to the parser is

192.168.107.132:12260 -> 207.44.101.104:12260 to 48.70.67.223:940

So it really depends if this message is coming from real syslog source
or if it is coming from a "no_parse" source.

Evan.

Clements, Frank wrote:
...
Hello Everyone!
I've been trying to get some custom patterns put together to do some log
correlation and I'm having one hell of a time getting a working
pattern.  I think I need a second, third, forth set of eyes on this ...
Any help is appreciated!
Message: Nov 25 12:02:27 GENERATED NAT-UDP-C: 192.168.107.132:12260 ->
207.44.101.104:12260 to 48.70.67.223:940
Pattern: @IPvANY:.dict.insideAddr@:@NUMBER:.dict.insidePort@ ->
@IPvANY:.dict.outsideAddr@:@NUMBER:.dict.outsidePort@ to
@IPvANY:.dict.destAddr@:@NUMBER:.dict.destPort@
I've looked at a few examples from the community patterns, but nothing
in this stands out as being "wrong".
Thanks
-
Frank W Clements
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq
--
Evan Rempel                               erempel@uvic.ca
Senior Systems Administrator                 250.721.7691
Unix Services, University Systems, University of Victoria
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq