[syslog-ng] Suppress almost-identical messages?

[Fabien Wernli]

Hi,

On Thu, May 22, 2014 at 01:55:58PM -0400, Evade Flow wrote:
> Does syslog-ng support suppression of almost-but-not-quite identical
> messages? It would be nice to see something like this in the logs:

Here's the way I'd try doing it, based on your example:

1) Use filters to separate the "myapp: Battery voltage is N volts" message
   flow from all other messages.
2) Tag the "Regular flow" of messages using e.g. the tag "keep"
   Optionally tag the other stream using "drop"
3) Parse the "myapp" messages using a patterndb parser, correlating all
   messages from the same host, then trigger an action upon timeout or
   reaching threshold (use context-length macro). The generated message can
   contain useful information from the context, e.g. number of similar
   messages (CONTEXT_LENGTH), average of voltages (would probably need some
   template function acrobatics), etc.
4) Tag the action generated message using "keep"
5) Filter out all messages not having the tag "keep". This way the
   individual "myapp" messages won't be logged. Only the "regular flow" of
   messages and the triggered messages will.