[syslog-ng] logic and duplicate suppression

[Balazs Scheidler]

On Wed, 2010-07-28 at 10:04 -0500, John Kristoff wrote:
> I have a couple of scenarios where I'm looking to enhance how I handle
> and process some logs.  I'm looking for suggestions on what my options
> are, but maybe these are potential feature requests?
> 
> 1. In using a parser (cvs or the patterndb), I'd like to use some
> conditionals based on a resultant macro value.  So for example, if I
> have an sshd authentication log message with a source address in a
> macro and that address is contained w/in a specific prefix, I'd like to
> handle that message differently.  Perhaps not log it all or set another
> MACRO to a certain value.

I had a similar idea for a while and as an incentive for you to try the
latest-greatest stuff, I've implemented it in OSE 3.2:

commit b3f4c03473a0f77bf7d87abf3f00b46e035bbbe8
Author: Balazs Scheidler <bazsi at balabit.hu>
Date:   Thu Jul 29 09:59:53 2010 +0200

    rewrite: implement condition() option for rewrite expressions
    
    This patch implements condition() option for rewrite expression, which
    makes it possible to only apply a given reply rule if the
    message matches the filter.
    
    For example:
    
        set("something new" condition(facility(auth)));


> 2. I'd like to be able to suppress duplicate messages even if they are
> not necessarily contiguous at the destination.  So for example, if I
> have a SSH client that generates a log of its SSH client protocol
> and software, I don't need to see that over and over again (e.g. as
> you might commonly see today in SSH brute force attacks).

This is more difficult. The sane way of doing this is to keep state on a
per-host basis, which is the area of correllation. Of course this is on
the radar for syslog-ng, but we're not there yet.

Doing simply on the source side is not going to work as multiple
"source" hosts can appear on the same connection.

-- 
Bazsi