RE: [syslog-ng] prune identical messages
But the message repeating does not give you any new information so it is a waste of diskspace to store it. Because It is just as helpfull to say foo1: ssh connection from 129.257.10.4 foo1: 2,348 duplicate messages suppressed then to say foo1: ssh connection from 129.257.10.4 foo1: ssh connection from 129.257.10.4 foo1: ssh connection from 129.257.10.4 foo1: ssh connection from 129.257.10.4 ... foo1: ssh connection from 129.257.10.4 foo1: ssh connection from 129.257.10.4 foo1: ssh connection from 129.257.10.4 foo1: ssh connection from 129.257.10.4 I only want to throttle the part that writes the message to the disk. -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu]On Behalf Of Valdis.Kletnieks@vt.edu Sent: March 27, 2006 2:59 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] prune identical messages On Mon, 27 Mar 2006 14:25:51 EST, Richard Legault said:
How can I prevent a log from being written that is identical to the log message that immediately preceded it. I would like to throttle those messages so that they can only be printed once every 10 minutes, those occurring between would simply be dropped.
You *don't* want to simply drop them. For instance, there's a *big* difference between: foo1: ssh connection from 129.257.10.4 and foo1: ssh connection from 129.257.10.4 foo1: 2,348 duplicate messages suppressed Similarly, how would your response differ for: frobozz13: Correctable ECC error detected on board 4, SIMM 7. and frobozz13: Correctable ECC error detected on board 4, SIMM 7. frobozz13: 1,438,598 duplicate messages suppressed
On Mon, 27 Mar 2006 15:26:41 EST, Richard Legault said:
But the message repeating does not give you any new information so it is a wa ste of diskspace to store it. Because It is just as helpfull to say foo1: ssh connection from 129.257.10.4 foo1: 2,348 duplicate messages suppressed
*exactly*. But what you asked for was (your words now): "those occurring between would simply be dropped." Producing a "duplicate messages suppressed" is *not* "simply dropping". Also, notice that if you have an interleaved series of 2 or more message streams that are identical to themselves but not each other, you can get this: foo1: message 1 foo1: message 2 foo1: message 3 foo1: message 2 foo1: message 3 foo1: message 1 foo1: message 3 and so on - this requires keeping a copy of the last N messages rather than just the last single one to do it correctly. Proper choice of N to prevent being DoS'ed by an out-of-memory when flooded by a mass of non-identical messages is left as an exercise for the student....
participants (2)
-
Richard Legault
-
Valdis.Kletnieks＠vt.edu