[syslog-ng] repeated lines

[Balazs Scheidler]

On Wed, 2010-10-13 at 11:16 +0200, Peter Czanik wrote:
> Hello,
> 
> I'm just preparing patterns for uw-imapd. Problem:
> 
> Oct 12 16:54:12 ubuntu imapd[10555]: connect from 192.168.2.179
> (192.168.2.179)
> Oct 12 16:54:12 ubuntu imapd[10555]: imap service init from 192.168.2.179
> Oct 12 16:54:17 ubuntu imapd[10555]: Login failed user=asdf auth=asdf
> host=czp.localnet [192.168.2.179]
> Oct 12 16:54:20 ubuntu imapd[10555]: AUTHENTICATE PLAIN failure
> host=czp.localnet [192.168.2.179]
> Oct 12 16:54:20 ubuntu imapd[10555]: Login failed user=asdf auth=asdf
> host=czp.localnet [192.168.2.179]
> Oct 12 16:54:48 ubuntu imapd[10555]: Unexpected client disconnect, while
> reading line user=asdf host=czp.localnet [192.168.2.179]
> 
> As you can see, the line containing most information about the login
> failure is repeated twice:
> 
> Login failed user=asdf auth=asdf host=czp.localnet [192.168.2.179]
> 
> How can I make sure, that only one name value pair is generated from
> this event?

in this case you need to start a correllation context on the first
event, using a timeout that will hopefully cover the 2nd too.

and generate the resulting event at timeout.

e.g.

<rule context-scope="process" context-id="uw-imapd-login" context-timeout="10">
  <actions>
    <action trigger="timeout">
      <message>
        <values>...</values>
        <tags>...</tags>
      </message>
    </action>
  </actions>
</rule>

I think we'd need a nice copy mechanism, but right now you need to list 
all the name-value pairs in the <values> tag to get the desired result.

> 
> Bazsi: we talked about the importance of time on Monday: messages for a
> single event arrived in an 8 seconds time span...

that's not necessarily a problem. I think the real problem happens when
the timeout causes the number of parallel contexts to skyrocket. If a
correllation state is kept open for 7 days, but most of them close
within 1 minute, that's not a problem.

-- 
Bazsi