[syslog-ng] [PATCH 2/2] [dbparser] min and max message count condition in correlation actions

[Evan Rempel]

In your (new) example,

     <actions>
       <action if-msg-count-ge='2'>
         <message>
           <values>
             <value name="PROGRAM">event</value>
             <value name="MESSAGE">Hello event!</value>
           </values>
         </message>
       </action>
       <action if-msg-count-le='1'>
         <message>
           <values>
             <value name="PROGRAM">error</value>
             <value name="MESSAGE">Hello error!</value>
           </values>
         </message>
       </action>
     </actions>


There is only an allowance for the two cases <=1 and >-2

It might be nice to allow for multiple if-msg actions such that I could do something at 3 messages and then another at 15 messages etc.
What I see in our logs are warnings that are correlated and repeat, and they never get to an "error" according to the application, but
if there are more the 10 such warning, there is a problem state. Some construct that would allow

<action if_msg_count='$count ge 2">...
<action if_msg_count='($count % 10) eq 0'>...

so that I can put an expression into the value and do actions on all kinds of things.

In a different application I wrote, I used a "printf" style value. So '(%d %% 10) eq 0'
This allows for expressions that contain the value more than once '(%d le 5) or (%d ge 10)'.

All of this assumes that there is some way to keep a correlation open and activate multiple triggers to evaluate these actions, which may not be possible.

Just thinking out loud.

Evan.

________________________________________
From: syslog-ng-bounces at lists.balabit.hu [syslog-ng-bounces at lists.balabit.hu] On Behalf Of Gergely Nagy [algernon at balabit.hu]
Sent: Saturday, November 10, 2012 7:36 AM
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] [PATCH 2/2] [dbparser] min and max message count       condition in correlation actions

balint.kovacs at balabit.com writes:

> It was quite unintuitive to determine if a previous message exists in
> the correlation context as the basis of the decision whether an action
> should be run or not. This patch adds support for the min-messages and
> max-messages attributes of the action XML node, the action is run only,
> of the correlation context contains more or equal messages as the
> min-messages attribute and less or equal messages as the max-messages
> attribute.
>
> This way it is easy to determine if a context contains sufficient
> amount of messages for the event to be complete and run a different
> action if it is not, e.g. if 2 messages are needed for an event,
> this should emit an event message if there are enough messages and
> an error message if not.

Now this was confusing at first (and even after in-person explanation,
it is still a bit hard to wrap my head around it based on this
description). The naming is bad, sorry!

Just to clarify it to myself, what happens here, is that the patch
introduces two new attributes for the action tag. Since I can't explain
what they accomplish too well yet, I'd like to use an example:

Lets suppose we have an event we want to corellate from at least five
messages:

program: session=f00; login; username="user"
program: session=f00; task="something"
program: session=f00; status="ok"
program: session=f00; task="noop"
program: session=f00; logout

We want to make sure that whenever a corellation action fires, we have
all five messages (and we'll assume that the five messages we have are
the correct ones): for that, there's the new min-messages
attribute. When set to 5, it will only emit a message when there are
five or more messages in the context. Add this action to the closing
pattern, and missin accomplished.

It is most useful for closing patterns, but there's probably use case
for using it elsewhere too.

And to detect errors, we need a way to emit a message when we receive a
closing pattern, but don't have enough messages: that's what
max-messages does.

Lets say that the noop task did not happen, so when the logout message
arrives, we only have four messages in the context. If max-messages is
set to 4, then the action will trigger.

I find the naming unintuitive, though, it's kind of backwards...

How about if-msg-count-ge (min-messages) and if-msg-count-le
(max-messages)? I think that would make it slightly easier to understand
what they're used for.

Transforming your example, this:

>    <actions>
>      <action min-messages='2'>
>        <message>
>          <values>
>            <value name="PROGRAM">event</value>
>            <value name="MESSAGE">Hello event!</value>
>          </values>
>        </message>
>      </action>
>      <action max-messages='1'>
>        <message>
>          <values>
>            <value name="PROGRAM">error</value>
>            <value name="MESSAGE">Hello error!</value>
>          </values>
>        </message>
>      </action>
>    </actions>

...would turn into:

     <actions>
       <action if-msg-count-ge='2'>
         <message>
           <values>
             <value name="PROGRAM">event</value>
             <value name="MESSAGE">Hello event!</value>
           </values>
         </message>
       </action>
       <action if-msg-count-le='1'>
         <message>
           <values>
             <value name="PROGRAM">error</value>
             <value name="MESSAGE">Hello error!</value>
           </values>
         </message>
       </action>
     </actions>

I for one, would understand what this latter does without further
explanation. Provided I understood the intent, that is. :)

Perhaps we could even drop the if- prefix too, I'm not sure.

Any ideas or opinions?

--
|8]

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq