[syslog-ng] patterndb - user defined parsers

Martin Holste mcholste at gmail.com
Sun Nov 27 19:12:28 CET 2011


Ok, got it.  Yes, administration would be tough.  We currently handle
this with alerts in ELSA by parsing out the field we want to check and
then matching against it with a bunch of OR'd values like this:
+WINDOWS.eventid:538 username1 username2 username3

On Sun, Nov 27, 2011 at 11:58 AM, Evan Rempel <erempel at uvic.ca> wrote:
> Yes, I am aware of this, and it is good for generating the alerts when an event occurs, and I am planning on using
> this powerful feature as well, but it does not address my problem.
>
> The problem I am having is that I need to match on any login event where the login ID is one of 22 known
> administrator accounts. I don't want to have to augment the pattern for EVERY login type message to
> include the 22 different patterns. What I would like to do is may my own parser that would have
> the 22 known login IDs and then use that parser in the rest of the patterndb.
>
> This type of structure has already been applied to the syslog-ng configuration by adding the BLOCK { }; construct.
>
> Also, when a sysadmin leaves our group, or we hire new staff, I can just update my own parser for the new/removed
> sysadmin login ID and my infrastructure continues to work well.
>
> Evan
> ________________________________________
> From: syslog-ng-bounces at lists.balabit.hu [syslog-ng-bounces at lists.balabit.hu] On Behalf Of Martin Holste [mcholste at gmail.com]
> Sent: Sunday, November 27, 2011 9:47 AM
> To: Syslog-ng users' and developers' mailing list
> Subject: Re: [syslog-ng] patterndb - user defined parsers
>
> Sorry, see Bazsi's blog post on correlation, conditions, and contexts
> here: http://bazsi.blogs.balabit.com/2010/10/syslog-ng-correllation-updated/
> .
>
> On Sun, Nov 27, 2011 at 11:38 AM, Evan Rempel <erempel at uvic.ca> wrote:
>> I'm not sure what you mean by this.
>> I can not find anything on "conditional" for the patterndb. Do you mean that patterndb parses the login ID, and then
>> use the syslog-ng pattern matching to match that login ID to those of our system administrators?
>>
>> ________________________________________
>> From: syslog-ng-bounces at lists.balabit.hu [syslog-ng-bounces at lists.balabit.hu] On Behalf Of Martin Holste [mcholste at gmail.com]
>> Sent: Sunday, November 27, 2011 8:51 AM
>>
>> Have you tried to solve this using the current conditionals available?
>>
>> On Sun, Nov 27, 2011 at 12:27 AM, Evan Rempel <erempel at uvic.ca> wrote:
>>>
>>> It would be useful to permit users to define parsers in the patterndb.
>>> For example, in our environment, by policy we user a special set and order of characters of our
>>> administrators log into hosts and administer them. It would be useful to define a parser of
>>>
>>> @SYSADMIN@ that would match only our sysadmin accounts.
>>> We could then use this parser in the patterndb to take some action such as sending
>>> a message to the administrators about the event.
>>>
>>> Another example would be to create parser for @LOCALIP@ that matches my organizaions IP space.
>>> That way a set of rules can be defined using @LOCALIP@ for some kind of alerting, and then any
>>> organization could redifine the @LOCALIP@ and use all of the goodness that some third party had created for
>>> monitoring logs like an intrusion protection system.
>>>
>>> Current parsers can be described as
>>>
>>> QSTRING
>>>  - match opening char
>>>  - while not closing char, keep looking
>>>
>>> ESTRING
>>>  - while not end string, keep looking
>>>
>>> NUMBER
>>>  - while digit keep looking
>>>
>>> So it seems that general parsers could be constructed  with two styles of matching, and
>>> then concatenating the together.
>>>
>>> 1. While in set of characters [some list of characters]
>>> 2. While not in set of characters [some list of characters]
>>>
>>> I would call these
>>> INSET to match 1 or more of a set of characters, unless a #-# were specified, then a minimum to a maximum would be required.
>>> OUTSET to match 1 or more of anything except the characters, unless a #-# were specified, then a minimum to a maximum would be required.
>>> (perhaps a count of + or * could be used to specify 1 or more and 0 or more respectively)
>>>
>>> and then limit the count of such occurrences so that you could build the @IPv4@ parser as
>>>
>>> @INSET::123456789*1@@INSET::0123456789:0-2 at .@INSET::123456789:1@@INSET::0123456789:0-2 at .@INSET::123456789:1@@INSET::0123456789:0-2 at .@INSET::123456789:1@@INSET::0123456789:0-2@
>>>
>>> and @NUMBER@ would be
>>> @INSET::123456789:1@@INSET::0123456789@
>>>
>>> @FLOAT@ would be
>>> @INSET::0123456789.@
>>>
>>> Then a user could make
>>> <parser name="THOUSAND">@INSET::,:0-1@@INSET::0123456789:3@</parser>
>>> <parser name="MONEY">$@INSET::123456789:1-3@@THOUSAND:::*@. at INSET::0123456789:2@
>>>
>>> This is kind of like inventing regular expressions :-(
>>>
>>> I'm not sure how well this fits into the radix tree matching structure, but I wanted to start this discussion.
>>>
>>> Given the MONEY example, I think it is obvious that there needs to be a way to specify repeating groups of "something"
>>>
>>> Let the discussion begin!
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>


More information about the syslog-ng mailing list