Try it without the * -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Evan Baer Sent: 23 February 2010 14:57 To: Zoltán Pallagi Cc: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] problem with filtering in syslog-ng The ntpd lines are still falling through. Should I include the final() syntax on all my log lines? filter not_ntpd { not program("ntpd*"); }; log { source(src); filter(f_notice); filter(f_not_authpriv); filter(not_ntpd); destination(messages); flags(final); }; log { source(src); filter(f_kern); filter(f_debug); destination(messages); }; log { source(src); filter(f_lpr); filter(f_info); destination(messages); }; log { source(src); filter(f_mail); filter(f_crit); destination(messages); }; Here is an example of what we still see in the logs: Feb 23 09:55:50 hobbit-shn1f0503 ntpd[729]: Listening on interface #2 lo0, 127.0.0.1#123 Enabled Feb 23 09:55:50 hobbit-shn1f0503 ntpd[729]: Listening on routing socket on fd #23 for interface updates On Tue, Feb 23, 2010 at 9:40 AM, Zoltán Pallagi <pzolee@balabit.hu> wrote:

Hi Evan,

use the final flag in this log path. For example:

log { source(src); filter(f_kern); filter(f_debug); filter(not_ntpd); destination(messages); flags(final);  };

For more information see our documentation: http://www.balabit.hu/dl/html/syslog-ng-v3.0-guide-admin-en.html/ch08s03.htm...

Evan Baer wrote:

...

Hello,

I'm trying to exclude ntpd update messages from /var/log/messages.

I have created a filter like so:    filter not_ntpd { not program("ntpd*"); };

And implemented it in my log lines:

log { source(src); filter(not_ntpd); destination(messages); }; log { source(src); filter(f_notice); filter(f_not_authpriv); filter(not_ntpd); destination(messages); }; log { source(src); filter(f_kern); filter(f_debug); filter(not_ntpd); destination(messages); }; log { source(src); filter(f_lpr); filter(f_info); destination(messages); }; log { source(src); filter(f_mail); filter(f_crit); destination(messages); };

Yet the lines for ntpd still seem to pass through to the logfile.

Feb 23 09:20:48 magpie-shn1e1324 ntpd[37397]: synchronized to 10.2.253.22, stratum 1 Feb 23 09:20:49 magpie-shn1f1318 ntpd[729]: kernel time sync status change 2001

Any thoughts?    -- Evan

______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

-- pzolee

______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

log { source(src); filter(f_notice); filter(f_not_authpriv); filter(not_ntpd); destination(messages); flags(final); };

Question for everyone: If you have multiple filters in a single log statement does syslog-ng "OR" them or does it "AND" them? The former might explain this behaviour... Joe. -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Evan Baer Sent: 23 February 2010 15:59 To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] problem with filtering in syslog-ng Same result, the ntpd lines still make it through. On Tue, Feb 23, 2010 at 10:09 AM, Fegan, Joe <Joe.Fegan@hp.com> wrote:

Try it without the *

-----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Evan Baer Sent: 23 February 2010 14:57 To: Zoltán Pallagi Cc: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] problem with filtering in syslog-ng

The ntpd lines are still falling through.  Should I include the final() syntax on all my log lines?

filter not_ntpd { not program("ntpd*"); };

log { source(src); filter(f_notice); filter(f_not_authpriv); filter(not_ntpd); destination(messages); flags(final); }; log { source(src); filter(f_kern); filter(f_debug); destination(messages); }; log { source(src); filter(f_lpr); filter(f_info); destination(messages); }; log { source(src); filter(f_mail); filter(f_crit); destination(messages); };

Here is an example of what we still see in the logs:

Feb 23 09:55:50 hobbit-shn1f0503 ntpd[729]: Listening on interface #2 lo0, 127.0.0.1#123 Enabled Feb 23 09:55:50 hobbit-shn1f0503 ntpd[729]: Listening on routing socket on fd #23 for interface updates

On Tue, Feb 23, 2010 at 9:40 AM, Zoltán Pallagi <pzolee@balabit.hu> wrote:

...

Hi Evan,

use the final flag in this log path. For example:

log { source(src); filter(f_kern); filter(f_debug); filter(not_ntpd); destination(messages); flags(final);  };

For more information see our documentation: http://www.balabit.hu/dl/html/syslog-ng-v3.0-guide-admin-en.html/ch08s03.htm...

Evan Baer wrote:

...

Hello,

I'm trying to exclude ntpd update messages from /var/log/messages.

I have created a filter like so:    filter not_ntpd { not program("ntpd*"); };

And implemented it in my log lines:

log { source(src); filter(not_ntpd); destination(messages); }; log { source(src); filter(f_notice); filter(f_not_authpriv); filter(not_ntpd); destination(messages); }; log { source(src); filter(f_kern); filter(f_debug); filter(not_ntpd); destination(messages); }; log { source(src); filter(f_lpr); filter(f_info); destination(messages); }; log { source(src); filter(f_mail); filter(f_crit); destination(messages); };

Yet the lines for ntpd still seem to pass through to the logfile.

Feb 23 09:20:48 magpie-shn1e1324 ntpd[37397]: synchronized to 10.2.253.22, stratum 1 Feb 23 09:20:49 magpie-shn1f1318 ntpd[729]: kernel time sync status change 2001

Any thoughts?    -- Evan

______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

-- pzolee

______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

On Tue, 2010-02-23 at 16:21 -0500, Jerry Bell wrote:

I experimentally confirmed that multiple filters are treated as "AND" in a log{} statement.

and this is the desired behaviour. -- Bazsi

I pared down the log() lines and it worked and filtered as expected. There was a trailing entry, at the end of the file, which seemed to be causing the problem. Thanks for the help today! On Tue, Feb 23, 2010 at 5:50 PM, Jerry Bell <jerry@nrdx.com> wrote:

I am suspicious that the program() filter is not working as expected on these logs.  Can you try to create a separate log file for just ntpd logs like this?

filter ntpd {program("ntpd"); }; destination d_ntpd { file("/var/log/ntpdtest"); }; log { source(src); filter(ntpd); destination(d_ntpd); };

That would tell us whether the program filter is actually interpreting the program field properly for these logs.

Jerry

On 2/23/2010 10:58 AM, Evan Baer wrote:

...

Same result, the ntpd lines still make it through.

On Tue, Feb 23, 2010 at 10:09 AM, Fegan, Joe<Joe.Fegan@hp.com>  wrote:

...

Try it without the *

-----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Evan Baer Sent: 23 February 2010 14:57 To: Zoltán Pallagi Cc: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] problem with filtering in syslog-ng

The ntpd lines are still falling through.  Should I include the final() syntax on all my log lines?

filter not_ntpd { not program("ntpd*"); };

log { source(src); filter(f_notice); filter(f_not_authpriv); filter(not_ntpd); destination(messages); flags(final); }; log { source(src); filter(f_kern); filter(f_debug); destination(messages); }; log { source(src); filter(f_lpr); filter(f_info); destination(messages); }; log { source(src); filter(f_mail); filter(f_crit); destination(messages); };

Here is an example of what we still see in the logs:

Feb 23 09:55:50 hobbit-shn1f0503 ntpd[729]: Listening on interface #2 lo0, 127.0.0.1#123 Enabled Feb 23 09:55:50 hobbit-shn1f0503 ntpd[729]: Listening on routing socket on fd #23 for interface updates

______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

On Tue, 2010-02-23 at 14:57 -0800, Lance Laursen wrote:

You are defining many log statements for the same source and destination pair. This is unnecessary and likely causing problems. Since you have one source (src) that you want to apply many filters to prior to writing out to (message), you should be consolidating all of your source(src) destination(messages) into one log command:

log { source(src); filter(not_ntpd); filter(f_notice); filter(f_not_authpriv); filter(f_kern); filter(f_debug); filter(f_lpr); filter(f_info); filter(f_mail); filter(f_crit); destination(messages); };

If you want to log to files like /var/log/mail and /var/log/kern.log AS WELL as /var/log/messages, you can use that statement as-is. If you just want everything from source(src) to log to /var/log/messages, put a flags(final); at the end of your log statement, or just simplify and consolidate your log statement to: log { source(src); filter(not_ntpd); filter(f_not_authpriv); destination(messages); };

Remember the placement of your log statement containing flags(final); above or below your other log statements matters. If syslog-ng matches something and then sees a flags(final);, it will no longer write that log entry out to any other file afterwards.

It might also be useful to know that syslog-ng starting with 3.0 supports embedded log statements, e.g. you can do things like: log { source(src); filter(f_filter1); log { filter(f_filter2); destination(d_dst1); }; log { filter(f_filter3); destination(d_dst2); }; }; e.g. d_dst1 would receive filter1 && filter2 d_dst2 would receive filter1 && filter3 this will reduce the number of "final" flags needed for a given configuration, and also reduce the number of operations needed to process a message. -- Bazsi