[syslog-ng] Converting filtering from 2.1 to 3.0?

[Matthew Hall]

On Mon, Sep 20, 2010 at 04:18:03PM -0400, Worsham, Michael wrote:
> We are running syslog-ng v2.1 in a multiple client-server 
> configuration. Due to new encryption requirements, we planning to 
> upgrade to v3.0.8 soon and use the TLS capabilities. Upon one of our 
> test servers, I discovered that the following filtering works just 
> fine under syslog-ng v2.1 and blocks the messages from reaching the 
> remote syslog-ng server:
> 
> filter M_audit   { not match("Audit daemon rotating log files"); };
> filter M_snmp   { not match("snmpd"); };
> filter M_repeat  { not match("last message repeated"); };
> filter M_stats    { not match("Log statistics"); };
> log { source(s_localhost);
>        filter(M_audit);
>        filter(M_snmp);
>        filter(M_repeat);
>        filter(M_stats);
>        destination(d_general); # Remote Syslog server
> };
> 
> However under syslog-ng v3.0.8, these messages are getting passed over 
> to the remote syslog server. What do I need to do to strip these 
> messages from getting passed through? I tried using "not message" but 
> that doesn't seem to work.
> 
> -- Michael

Hello Michael,

I think you are witnessing a side effect of the following change:

http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.1-guide-admin-en.html/reference_macros.html

MSG or MESSAGE

Description: Text contents of the log message without the program name 
and pid. Note that this has changed in syslog-ng version 3.0; in earlier 
versions this macro included the program name and the pid. In syslog-ng 
3.0, the MSG macro became equivalent with the MSGONLY macro. The program 
name and the pid together are available in the MSGHDR macro.

http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.1-guide-admin-en.html/reference_filters.html

match()
Synopsis: 	match(regexp)

Description: Match a regular expression to the headers and the message 
itself (i.e., the values returned by the MSGHDR and MSG macros). Note 
that in syslog-ng version 2.1 and earlier, the match() filter was 
applied only to the text of the message, excluding the headers. This 
functionality has been moved to the message() filter. To limit the scope 
of the match to a specific part of the message (identified with a 
macro), use the match(regexp value("MACRO")) syntax. Do not include the 
$ sign in the parameter of the value() option.

I suggest checking to be sure you are matching against the right macro.

You can change the macro using "value" arguments as shown here:

http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.1-guide-admin-en.html/configuring_filters.html

You might also want to add -d and/or -v options to launch the daemon 
with debugging enabled to see how the messages get processed to see what 
filters are matching or failing and why.

Good Luck!
Matthew.