[syslog-ng] PatternDB with CiscoSNMP Module

Nik Ambrosch nik at ambrosch.com
Wed May 3 16:09:06 UTC 2017 

Just a heads up - Cisco Nexus devices have a different format than normal ios devices, it’s kind of obnoxious.

2017 Apr 26 18:04:43


> On May 3, 2017, at 2:26 AM, Fekete, Róbert <robert.fekete at balabit.com> wrote:
> 
> Hi, you can find the description of junctions and channels here: https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/junctions.html <https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/junctions.html>
> 
> and here: https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/embedded-objects.html <https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/embedded-objects.html>
> 
> On Wed, May 3, 2017 at 1:54 AM, Schoonover, Mark E HHHH <Mark.Schoonover at cigna.com <mailto:Mark.Schoonover at cigna.com>> wrote:
> Thanks Bazsi,
> 
>  
> 
> It is a challenge that’s for sure. Looking at the code, I think it’ll parse a Cisco syslog message like this:
> 
>  
> 
> Apr 26 14:04:45 hostname 565049: Apr 26 18:04:43.476 UTC: %EARL-DFC1-1-EXCESSIVE_PARITY_ERROR: EARL 0: Parity error detected in VRAM
> 
>  
> 
> This snippet
> 
>             csv-parser(delimiters(chars('-')) template("$3")
> 
>                        columns('.cisco.facility', '.cisco.severity', '.cisco.mnemonic'));
> 
>  
> 
> into:
> 
>  
> 
> .cisco.facility = EARL
> 
> .cisco.severity = DFC1
> 
> .cisco.mnemonic = 1
> 
>  
> 
> I’m not familiar with channels and junctions and didn’t find anything in the OSE admin manual. Possibly I’m not fully understanding either.
> 
>  
> 
> Regards,
> 
>  
> 
> Mark Schoonover – KA6WKE
> 
> Infrastructure Engineering Manager, Splunk Architect
> 
> ENE   : Tools, Instrumentation and Common Services Team
> 
> Office: 32.8697° N, 116.9711° W
> 
> Phone : 770-261-7934 <tel:(770)%20261-7934>
> Email : mark.schoonover at cigna.com <mailto:mark.schoonover at cigna.com>
> HPSM Team: ENE NMS Engineering
> 
>  
> 
> Confidential, unpublished property of CIGNA. Do not duplicate or distribute. Use and distribution limited solely to authorized personnel. © Copyright 2017 CIGNA
> 
>  
> 
>  
> 
> From: syslog-ng [mailto:syslog-ng-bounces at lists.balabit.hu <mailto:syslog-ng-bounces at lists.balabit.hu>] On Behalf Of Scheidler, Balázs
> Sent: Monday, May 01, 2017 11:57 PM
> To: Syslog-ng users' and developers' mailing list
> Subject: Re: [syslog-ng] PatternDB with CiscoSNMP Module
> 
>  
> 
> Hi,
> 
> I am in the process of writing a cisco-parser() myself, which uses a combination of csv-parser() and regexps, pattern-db is not very well suited for breaking apart Cisco like conditional structure (which depends on various settings on the cisco side).
> 
> While handling various settings (service sequence-numbers, service timestamps, logging origin-id, etc), it achieves about 37k msg/sec. What is your target number with regards to performance?
> 
> Here's my stuff:
> https://github.com/balabit/syslog-ng/pull/1456 <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_balabit_syslog-2Dng_pull_1456&d=DwMFaQ&c=WlnXFIBCT3pmNh_w8hYeLw&r=7hmZ2rDjdhHVbv2tovCmuIwqUlXIJdVXqIASPRODb2U&m=s7SsNLIK6w8eENvA6fSVfIY_hmH_Ao37Vfkb8LMT5go&s=tBE35b_4ndD7X1zSxFEcMXgnwGM1GZ9VB3oIUGF-vtU&e=>
> 
> -- 
> Bazsi
> 
>  
> 
> On Tue, May 2, 2017 at 2:17 AM, Schoonover, Mark E HHHH <Mark.Schoonover at cigna.com <mailto:Mark.Schoonover at cigna.com>> wrote:
> 
> Thanks for reading,
> 
>  
> 
> I’m in the process of creating a log to trap for Cisco devices. I have it working for facilities like this: TRINITY-2-TRINITY_SYSLOG_CRIT with this pattern:
> 
>  
> 
> @ESTRING::%@@ESTRING:.cisco.Facility:-@@ESTRING:.cisco.Severity:-@@ESTRING:.cisco.MsgName::@ @ANYSTRING:.cisco.MsgText@
> 
>  
> 
> Now there are other facilities with an additional hyphen before the severity: EARL-DFC1-1-EXCESSIVE_PARITY_ERROR:
> 
>  
> 
> Apr 26 14:04:45 hostname 565049: Apr 26 18:04:43.476 UTC: %EARL-DFC1-1-EXCESSIVE_PARITY_ERROR: EARL 0: Parity error detected in VRAM'
> 
>  
> 
> which parses into:
> 
>  
> 
> Testing message: program='EARL-DFC1-1-EXCESSIVE_PARITY_ERROR' message='Apr 26 14:04:45 hostname 565049: Apr 26 18:04:43.476 UTC: %EARL-DFC1-1-EXCESSIVE_PARITY_ERROR: EARL 0: Parity error detected in VRAM'
> 
> Match name='.classifier.rule_id', value='09944c71-95eb-4bc0-8575-936931d85715', expected='09944c71-95eb-4bc0-8575-936931d85715'
> 
> Wrong match name='.cisco.Facility', value='EARL', expected='EARL-DFC1'
> 
> Wrong match name='.cisco.Severity', value='DFC1', expected='1'
> 
> Wrong match name='.cisco.MsgName', value='1-EXCESSIVE_PARITY_ERROR', expected='EXCESSIVE_PARITY_ERROR'
> 
> Match name='.cisco.MsgText', value='EARL 0: Parity error detected in VRAM', expected='EARL 0: Parity error detected in VRAM'
> 
>  
> 
> I’ve tried a pattern that would parse EARL and DFC1 into separate variables, then append them into .cisco.Facility – which didn’t work. Still that leaves the pattern hardcoded to just two hypens. I’ve also tried hard coding EARL-DFC1 into the pattern, then use value to set .cisco.Facility correctly. This approach works but being hardcoded, another Facility with embedded hyphens will fail. Ideally I’d like to have a pattern that matches on the last hyphen before the severity and capture that. I could parse using a regexp and setting $1, $2, $3, and $4 to the appropriate variables but I’m concerned about performance issues – I’m thinking this is a last resort solution.
> 
>  
> 
> Any thoughts on how to proceed?
> 
>  
> 
> Regards,
> 
>  
> 
> Mark Schoonover – KA6WKE
> 
> Infrastructure Engineering Manager, Splunk Architect
> 
> ENE   : Tools, Instrumentation and Common Services Team
> 
> Office: 32.8697° N, 116.9711° W
> 
> Phone : 770-261-7934 <tel:(770)%20261-7934>
> Email : mark.schoonover at cigna.com <mailto:mark.schoonover at cigna.com>
> HPSM Team: ENE NMS Engineering
> 
>  
> 
> Confidential, unpublished property of CIGNA. Do not duplicate or distribute. Use and distribution limited solely to authorized personnel. © Copyright 2017 CIGNA
> 
>  
> 
>  
> 
> ------------------------------------------------------------------------------
> CONFIDENTIALITY NOTICE: If you have received this email in error,
> please immediately notify the sender by e-mail at the address shown.  
> This email transmission may contain confidential information.  This 
> information is intended only for the use of the individual(s) or entity to 
> whom it is intended even if addressed incorrectly.  Please delete it from 
> your files if you are not the intended recipient.  Thank you for your 
> compliance.  Copyright (c) 2017 Cigna
> ==============================================================================
> 
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.balabit.hu_mailman_listinfo_syslog-2Dng&d=DwMFaQ&c=WlnXFIBCT3pmNh_w8hYeLw&r=7hmZ2rDjdhHVbv2tovCmuIwqUlXIJdVXqIASPRODb2U&m=s7SsNLIK6w8eENvA6fSVfIY_hmH_Ao37Vfkb8LMT5go&s=4rnwzztJes-sEVyNI37_YvvyrlHkRLSwjQGprO-fmX0&e=>
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.balabit.com_support_documentation_-3Fproduct-3Dsyslog-2Dng&d=DwMFaQ&c=WlnXFIBCT3pmNh_w8hYeLw&r=7hmZ2rDjdhHVbv2tovCmuIwqUlXIJdVXqIASPRODb2U&m=s7SsNLIK6w8eENvA6fSVfIY_hmH_Ao37Vfkb8LMT5go&s=k_tNOdB-fJ6oAIfsJ6wU1JVQv5cTPockc95Diky6exc&e=>
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <https://urldefense.proofpoint.com/v2/url?u=http-3A__www.balabit.com_wiki_syslog-2Dng-2Dfaq&d=DwMFaQ&c=WlnXFIBCT3pmNh_w8hYeLw&r=7hmZ2rDjdhHVbv2tovCmuIwqUlXIJdVXqIASPRODb2U&m=s7SsNLIK6w8eENvA6fSVfIY_hmH_Ao37Vfkb8LMT5go&s=rNRCDCuEpMjwyl9gta5U2p59odVIqTgghc5xzMfOU_k&e=>
> 
>  
> 
> ------------------------------------------------------------------------------
> CONFIDENTIALITY NOTICE: If you have received this email in error,
> please immediately notify the sender by e-mail at the address shown.  
> This email transmission may contain confidential information.  This 
> information is intended only for the use of the individual(s) or entity to 
> whom it is intended even if addressed incorrectly.  Please delete it from 
> your files if you are not the intended recipient.  Thank you for your 
> compliance.  Copyright (c) 2017 Cigna
> ==============================================================================
> 
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng>
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng>
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>
> 
> 
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170503/ddd19b32/attachment-0001.html>

More information about the syslog-ng mailing list