[syslog-ng] prune identical messages

[Valdis.Kletnieks at vt.edu]

On Mon, 27 Mar 2006 15:26:41 EST, Richard Legault said:
> But the message repeating does not give you any new information so it is a wa
ste of diskspace to store it.
> Because It is just as helpfull to say 
> foo1: ssh connection from 129.257.10.4
> foo1: 2,348 duplicate messages suppressed

*exactly*.  But what you asked for was (your words now):

"those occurring between would simply be dropped."

Producing a "duplicate messages suppressed" is *not* "simply dropping".

Also, notice that if you have an interleaved series of 2 or more message
streams that are identical to themselves but not each other, you can get this:

foo1: message 1
foo1: message 2
foo1: message 3
foo1: message 2
foo1: message 3
foo1: message 1
foo1: message 3

and so on - this requires keeping a copy of the last N messages rather than
just the last single one to do it correctly.  Proper choice of N to prevent
being DoS'ed by an out-of-memory when flooded by a mass of non-identical
messages is left as an exercise for the student....
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 228 bytes
Desc: not available
Url : http://lists.balabit.hu/pipermail/syslog-ng/attachments/20060327/9a2c711a/attachment.pgp