I don't believe that syslog-ng can do that, but I use SEC (simple event correlator) for that purpose.


On Thu, May 22, 2014 at 10:55 AM, Evade Flow <evadeflow@gmail.com> wrote:
I'm trying to clean up the ad hoc logging in a collection of applications started on an embedded device using the daemon utility. Example:

   /usr/bin/daemon --name=myapp --output=user.debug --chdir=/MyApp -- /MyApp/bin/myapp

There are 10 different applications started this way, written by different individuals/teams, and our syslog-ng.conf file currently looks something like this:

@version:3.3.6
source s_local { system(); internal(); };
destination d_localfile { file("/var/log/messages" suppress(30)); };
log { source(s_local); destination(d_localfile); };

Because the --output=user.debug option to the daemon utility will cause the applications' stdout/stderr to be redirected to syslog, I'm a little worried about 'rogue' printf()'s filling up the logs with stuff like:

Jan  1 00:00:54 (none) myapp: Battery voltage is 11.07 volts
Jan  1 00:00:54 (none) myapp: Battery voltage is 11.10 volts
Jan  1 00:00:54 (none) myapp: Battery voltage is 11.07 volts
Jan  1 00:00:54 (none) myapp: Battery voltage is 11.14 volts
Jan  1 00:00:54 (none) myapp: Battery voltage is 11.13 volts
Jan  1 00:00:55 (none) myapp: Battery voltage is 11.09 volts
Jan  1 00:00:55 (none) myapp: Battery voltage is 11.14 volts
...

Does syslog-ng support suppression of almost-but-not-quite identical messages? It would be nice to see something like this in the logs:

Jan  1 00:55:11 myapp: Battery voltage is 11.07 volts
Jan  1 00:55:14 myapp: Last message 'Battery voltage is 1' repeated with nearly identical content 418 times, suppressed by syslog-ng on flahblargle

A few of my colleagues have suggested that this is probably more trouble than it's worth, and we should just fix the spammy app(s) rather than try to 'outsmart' them. And I think I agree with them, but... it was my idea to move to syslog-ng, so there'll be a bit more egg on my face than theirs if we accidentally allow a rogue printf() in a hot loop somewhere to escape into production. `:-}

Any advice/insight much appreciated, thanks!

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq