Dropping specific messages from Syslog-NG
Disclaimer: I'm not an expert in either Linux or syslog-ng. I'm also not a programmer :). That being said, I've built a syslog-ng --> Splunk deployment and I'm trying to drop unwanted log messages before they make their way to Splunk... so drop them at syslog-ng. There are going to be approx. 700 devices logging to this deployment (routers, switches, firewalls, IPAM, AAA, etc..) so I need to make sure whatever I do can scale easily and my teammates can help me manage it. I'm trying to drop messages that are similar to the below: Apr 5 06:02:17 x.x.x.x 2019 Apr 5 06:02:17 EDT: %USER-3-SYSTEM_MSG: NTP Receive dropping message: Received NTP control mode packet. Drop count:4200 - ntpd As well as: Apr 5 08:03:49 x.x.x.x Apr 5 2019 12:03:48.895 UTC: %SNMP-3-RESPONSE_DELAYED: processing GetNext of entPhysicalEntry.2.1065 (5739 msecs) Two different devices. I've tried literally dozens of methods, my latest is below and I can't figure out why we're still logging these messages: filter f_trash { match(SNMP-3-RESPONSE_DELAYED value(MSGHDR)); and match(NTP Receive dropping message value(MSG))}; log { source(s_sys); filter(f_trash); flags (final);}; Any help / constructive criticism would be greatly appreciated.
Hi, It seems your setup incorrectly parses the "syslog" messages. The latest syslog-ng versiosn have improved Cisco parsing code, so I suggest you try those first. In any case, I can see two problems with your filter:
filter f_trash { match(SNMP-3-RESPONSE_DELAYED value(MSGHDR)); and
match(NTP Receive dropping message value(MSG))};
1. it matches only messages with both "SNMP" and "NTP" strings, which is not what you seem to want. 2. "SNMP" seems to be in the MSG macro instead, although I can't be sure 3. use quotes in the match() argument Cheers
Thanks for the advice! I will fix the incorrect Boolean usage and add some quotes to my match statement. Chris Bell Network Engineer 352-273-9186 (w) ext 39186 -----Original Message----- From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Fabien Wernli Sent: Friday, April 5, 2019 8:59 AM To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] Dropping specific messages from Syslog-NG CAUTION! This email came from outside UF or UF Health. Exercise extra caution clicking links and opening attachments from any and all senders. ________________________________ Hi, It seems your setup incorrectly parses the "syslog" messages. The latest syslog-ng versiosn have improved Cisco parsing code, so I suggest you try those first. In any case, I can see two problems with your filter:
filter f_trash { match(SNMP-3-RESPONSE_DELAYED value(MSGHDR)); and
match(NTP Receive dropping message value(MSG))};
1. it matches only messages with both "SNMP" and "NTP" strings, which is not what you seem to want. 2. "SNMP" seems to be in the MSG macro instead, although I can't be sure 3. use quotes in the match() argument Cheers ______________________________________________________________________________ Member info: https://imsva91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2... Documentation: https://imsva91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f... FAQ: https://imsva91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f...
Thanks for the help on this, you're suggestions fixed both my issues. The final, working, config for dropping specific messages looks like this: filter f_trash { match('%SNMP-3-RESPONSE_DELAYED:' value(MSG)) or match ('NTP Receive dropping message:' value(MSG)); }; # Drop off the trash # log { source(s_sys); filter(f_trash); flags (final); }; Chris Bell Network Engineer 352-273-9186 (w) ext 39186 -----Original Message----- From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Bell, Christopher E. Sent: Friday, April 5, 2019 9:47 AM To: 'wernli@in2p3.fr' <wernli@in2p3.fr>; Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] Dropping specific messages from Syslog-NG Thanks for the advice! I will fix the incorrect Boolean usage and add some quotes to my match statement. Chris Bell Network Engineer 352-273-9186 (w) ext 39186 -----Original Message----- From: syslog-ng <syslog-ng-bounces@lists.balabit.hu> On Behalf Of Fabien Wernli Sent: Friday, April 5, 2019 8:59 AM To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] Dropping specific messages from Syslog-NG CAUTION! This email came from outside UF or UF Health. Exercise extra caution clicking links and opening attachments from any and all senders. ________________________________ Hi, It seems your setup incorrectly parses the "syslog" messages. The latest syslog-ng versiosn have improved Cisco parsing code, so I suggest you try those first. In any case, I can see two problems with your filter:
filter f_trash { match(SNMP-3-RESPONSE_DELAYED value(MSGHDR)); and
match(NTP Receive dropping message value(MSG))};
1. it matches only messages with both "SNMP" and "NTP" strings, which is not what you seem to want. 2. "SNMP" seems to be in the MSG macro instead, although I can't be sure 3. use quotes in the match() argument Cheers ______________________________________________________________________________ Member info: https://imsva91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2... Documentation: https://imsva91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f... FAQ: https://imsva91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f... ______________________________________________________________________________ Member info: https://imsva91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=https%3a%2... Documentation: https://imsva91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f... FAQ: https://imsva91-ctp.trendmicro.com:443/wis/clicktime/v1/query?url=http%3a%2f...
participants (2)
-
Bell, Christopher E.
-
Fabien Wernli