[syslog-ng] patterndb - user defined parsers

[Fekete Róbert]

On Sunday, November 27, 2011 07:27 CET, Evan Rempel <erempel at uvic.ca> wrote: 
 
> 
> It would be useful to permit users to define parsers in the patterndb.
> For example, in our environment, by policy we user a special set and order of characters of our
> administrators log into hosts and administer them. It would be useful to define a parser of
> 
> @SYSADMIN@ that would match only our sysadmin accounts.
> We could then use this parser in the patterndb to take some action such as sending
> a message to the administrators about the event.
> 
> Another example would be to create parser for @LOCALIP@ that matches my organizaions IP space.
> That way a set of rules can be defined using @LOCALIP@ for some kind of alerting, and then any
> organization could redifine the @LOCALIP@ and use all of the goodness that some third party had created for
> monitoring logs like an intrusion protection system.
> 
Hi, it might not be exactly what you are after, but it is possible to use filters and template functions on parsed fields. 
Would it be possible to parse the IP with an IP parser, and then later in the logpath check its value with a regex/network filter? Or with a well-placed if template function?

Robert

> Current parsers can be described as
> 
> QSTRING 
>   - match opening char
>  - while not closing char, keep looking
> 
> ESTRING
>   - while not end string, keep looking
> 
> NUMBER
>   - while digit keep looking
> 
> So it seems that general parsers could be constructed  with two styles of matching, and
> then concatenating the together.
> 
> 1. While in set of characters [some list of characters]
> 2. While not in set of characters [some list of characters]
> 
> I would call these
> INSET to match 1 or more of a set of characters, unless a #-# were specified, then a minimum to a maximum would be required.
> OUTSET to match 1 or more of anything except the characters, unless a #-# were specified, then a minimum to a maximum would be required.
> (perhaps a count of + or * could be used to specify 1 or more and 0 or more respectively)
> 
> and then limit the count of such occurrences so that you could build the @IPv4@ parser as
> 
> @INSET::123456789*1@@INSET::0123456789:0-2 at .@INSET::123456789:1@@INSET::0123456789:0-2 at .@INSET::123456789:1@@INSET::0123456789:0-2 at .@INSET::123456789:1@@INSET::0123456789:0-2@
> 
> and @NUMBER@ would be
> @INSET::123456789:1@@INSET::0123456789@
> 
> @FLOAT@ would be
> @INSET::0123456789.@
> 
> Then a user could make
> <parser name="THOUSAND">@INSET::,:0-1@@INSET::0123456789:3@</parser>
> <parser name="MONEY">$@INSET::123456789:1-3@@THOUSAND:::*@. at INSET::0123456789:2@
> 
> This is kind of like inventing regular expressions :-(
> 
> I'm not sure how well this fits into the radix tree matching structure, but I wanted to start this discussion.
> 
> Given the MONEY example, I think it is obvious that there needs to be a way to specify repeating groups of "something"
> 
> Let the discussion begin!
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> 
>