I recall someone pointing out last year that syslogd has a feature where it can compress a burst of messages into a note like: Jan 10 17:01:06 loghost last message repeated 24 times While syslog-ng didn't have that. Is that still the case? I just ran into this with a mongo runaway program that logged several hundred messages a second, which filled up my log partition. If this feature still isn't in syslog-ng, is it one that we want? I could see it as a source option like "suppress-repeats", where it would spit out the original message followed by a tag, like this: Jan 10 17:25:39 loghost inetd[7729]: Connection from 10.24.1.168 [REPEATED 1024 TIMES] and maybe arguments to "suppress-repeats" like how many repeats should be printed before kicking in, or perhaps even a regex describing which parts of the log message to ignore before comparing it to the previous one. Thanks, -- Ed
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ed Ravin wrote:
I recall someone pointing out last year that syslogd has a feature where it can compress a burst of messages into a note like:
excerpt from syslog-ng-1.6.0rc4/contrib/syslog-ng.conf.doc filter f_postgres { not( (host("syslogdb") and facility(cron) and level(info)) or (facility(user) and level(notice) and ( match(" gethostbyaddr: ") or match("last message repeated ") ) ) - -- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu There's no place like 127.0.0.1 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (SunOS) iD4DBQFAAIJFUAgaF+Ymk8URAltxAJiO+I1vqjcyeVy0oLL4jWSkpk4HAJ0QUYdl DKxKsYs0YxTFHXYgAUscaA== =SBEy -----END PGP SIGNATURE-----
On Sat, Jan 10, 2004 at 05:52:54PM -0500, Asif Iqbal wrote:
Ed Ravin wrote:
I recall someone pointing out last year that syslogd has a feature where it can compress a burst of messages [...]
excerpt from syslog-ng-1.6.0rc4/contrib/syslog-ng.conf.doc
filter f_postgres { not( (host("syslogdb") and facility(cron) and level(info)) or (facility(user) and level(notice) and ( match(" gethostbyaddr: ") or match("last message repeated ") ) )
If I read that correctly, it removes messages that are unwanted, including the "last message repeated" from syslogd. I'm talking about having syslog-ng recognize that a stream of messages are duplicates, and issuing a friendlier version of "last message repeated" so that the duplicates are (mostly) suppressed. I suppose if syslog-ng had a variable for the last message string, then something like this could be done in a filter?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Ed Ravin wrote:
If I read that correctly, it removes messages that are unwanted, including the "last message repeated" from syslogd. I'm talking about having syslog-ng recognize that a stream of messages are duplicates, and issuing a friendlier version of "last message repeated" so that the duplicates are (mostly) suppressed.
I suppose if syslog-ng had a variable for the last message string, then something like this could be done in a filter?
I guess the you can try something like this filter f_repeat { match("last message repeated "); }; destination d_repeat { program("/usr/local/bin/check-repeat.pl"); }; check-repeat.pl compares only the filtered messages (each line at a time) with for example /var/adm/messages file. Then it will append the line to the /var/adm/messages file only if that line does not exist. - -- Asif Iqbal PGP Key: 0xE62693C5 KeyServer: pgp.mit.edu There's no place like 127.0.0.1 -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.2.1 (SunOS) iD8DBQFAAIjFUAgaF+Ymk8URAgZ7AKC529/IGYp9dj8HbQZZt1DldyVV4wCfTiAC r6e9KlI2mBRdg6KjTvGahyU= =NPYr -----END PGP SIGNATURE-----
participants (2)
-
Asif Iqbal
-
Ed Ravin