[syslog-ng] Filtering duplicate messages

[Balazs Scheidler]

On Fri, 2008-05-30 at 18:09 +1200, chris packham wrote:
> Updated patch - back to using suppress, fix potential segfault.
> 
> Implement "Last message repeated N times" functionality.
> 
> This behaviour can be enabled by adding the new "suppress(<num>)" option to
> an output configuration e.g.
> 
> destination tologfile { file("/var/log/messages" template(t) suppress(30)); };
> 
> As a log message is added to the queue it is remembered as the last message
> seen.
> 
> When a new message is added to the queue it is checked against the last
> message. If its contents are the same the message is dropped and a counter
> incremented.
> 
> A message summary indicating the value of the last message counter and a
> snippet of the message will be inserted into the log queue if a new message
> that differs is seen or if the configurable timeout period expires.
> ---

I have integrated your patches to OSE 2.1 with slight changes. These
were:
  * forward-ported it to OSE 2.1 as I don't intend to destabilize 2.0
with it
  * I've postponed the registration of the suppress timer to the first
dropped message, as adding/removing a timer for each message seems to be
a lot of overhead
  * I've added a separate stats counter type "suppressed" instead of
overloading "dropped", the latter is considered an error, and
suppressing messages is done by operator request.
  * cosmetic changes here and there.

My last patch is this:
http://git.balabit.hu/?p=bazsi/syslog-ng-2.1.git;a=commit;h=f39e60ac40e3a980039aae50b216e7afad5d80fa

I'd appreciate some testing of this functionality.



-- 
Bazsi