[syslog-ng] Suppress almost-identical messages?

[Orangepeel Beef]

I don't believe that syslog-ng can do that, but I use SEC (simple event
correlator) for that purpose.


On Thu, May 22, 2014 at 10:55 AM, Evade Flow <evadeflow at gmail.com> wrote:

> I'm trying to clean up the ad hoc logging in a collection of applications
> started on an embedded device using the daemon utility. Example:
>
>    /usr/bin/daemon --name=myapp --output=user.debug --chdir=/MyApp --
> /MyApp/bin/myapp
>
> There are 10 different applications started this way, written by different
> individuals/teams, and our syslog-ng.conf file currently looks something
> like this:
>
> @version:3.3.6
> source s_local { system(); internal(); };
> destination d_localfile { file("/var/log/messages" suppress(30)); };
> log { source(s_local); destination(d_localfile); };
>
> Because the --output=user.debug option to the daemon utility will cause
> the applications' stdout/stderr to be redirected to syslog, I'm a little
> worried about 'rogue' printf()'s filling up the logs with stuff like:
>
> Jan  1 00:00:54 (none) myapp: Battery voltage is 11.07 volts
> Jan  1 00:00:54 (none) myapp: Battery voltage is 11.10 volts
> Jan  1 00:00:54 (none) myapp: Battery voltage is 11.07 volts
> Jan  1 00:00:54 (none) myapp: Battery voltage is 11.14 volts
> Jan  1 00:00:54 (none) myapp: Battery voltage is 11.13 volts
> Jan  1 00:00:55 (none) myapp: Battery voltage is 11.09 volts
> Jan  1 00:00:55 (none) myapp: Battery voltage is 11.14 volts
> ...
>
> Does syslog-ng support suppression of almost-but-not-quite identical
> messages? It would be nice to see something like this in the logs:
>
> Jan  1 00:55:11 myapp: Battery voltage is 11.07 volts
> Jan  1 00:55:14 myapp: Last message 'Battery voltage is 1' repeated with
> nearly identical content 418 times, suppressed by syslog-ng on flahblargle
>
> A few of my colleagues have suggested that this is probably more trouble
> than it's worth, and we should just fix the spammy app(s) rather than try
> to 'outsmart' them. And I think I agree with them, but... it was my idea to
> move to syslog-ng, so there'll be a bit more egg on my face than theirs if
> we accidentally allow a rogue printf() in a hot loop somewhere to escape
> into production. `:-}
>
> Any advice/insight much appreciated, thanks!
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20140522/c1ce0dbc/attachment.htm