[syslog-ng] PatternDB with CiscoSNMP Module

Tue May 2 23:54:30 UTC 2017

Thanks Bazsi,

It is a challenge that’s for sure. Looking at the code, I think it’ll parse a Cisco syslog message like this:

Apr 26 14:04:45 hostname 565049: Apr 26 18:04:43.476 UTC: %EARL-DFC1-1-EXCESSIVE_PARITY_ERROR: EARL 0: Parity error detected in VRAM

This snippet
            csv-parser(delimiters(chars('-')) template("$3")
                       columns('.cisco.facility', '.cisco.severity', '.cisco.mnemonic'));

into:

.cisco.facility = EARL
.cisco.severity = DFC1
.cisco.mnemonic = 1

I’m not familiar with channels and junctions and didn’t find anything in the OSE admin manual. Possibly I’m not fully understanding either.

Regards,

Mark Schoonover – KA6WKE
Infrastructure Engineering Manager, Splunk Architect
ENE   : Tools, Instrumentation and Common Services Team
Office: 32.8697° N, 116.9711° W
Phone : 770-261-7934
Email : mark.schoonover at cigna.com<mailto:mark.schoonover at cigna.com>
HPSM Team: ENE NMS Engineering

From: syslog-ng [mailto:syslog-ng-bounces at lists.balabit.hu] On Behalf Of Scheidler, Balázs
Sent: Monday, May 01, 2017 11:57 PM
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] PatternDB with CiscoSNMP Module

Hi,
I am in the process of writing a cisco-parser() myself, which uses a combination of csv-parser() and regexps, pattern-db is not very well suited for breaking apart Cisco like conditional structure (which depends on various settings on the cisco side).
While handling various settings (service sequence-numbers, service timestamps, logging origin-id, etc), it achieves about 37k msg/sec. What is your target number with regards to performance?
Here's my stuff:
https://github.com/balabit/syslog-ng/pull/1456<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_balabit_syslog-2Dng_pull_1456&d=DwMFaQ&c=WlnXFIBCT3pmNh_w8hYeLw&r=7hmZ2rDjdhHVbv2tovCmuIwqUlXIJdVXqIASPRODb2U&m=s7SsNLIK6w8eENvA6fSVfIY_hmH_Ao37Vfkb8LMT5go&s=tBE35b_4ndD7X1zSxFEcMXgnwGM1GZ9VB3oIUGF-vtU&e=>

--
Bazsi

On Tue, May 2, 2017 at 2:17 AM, Schoonover, Mark E HHHH <Mark.Schoonover at cigna.com<mailto:Mark.Schoonover at cigna.com>> wrote:
Thanks for reading,

I’m in the process of creating a log to trap for Cisco devices. I have it working for facilities like this: TRINITY-2-TRINITY_SYSLOG_CRIT with this pattern:

@ESTRING::%@@ESTRING:.cisco.Facility:-@@ESTRING:.cisco.Severity:-@@ESTRING:.cisco.MsgName::@ @ANYSTRING:.cisco.MsgText@

Now there are other facilities with an additional hyphen before the severity: EARL-DFC1-1-EXCESSIVE_PARITY_ERROR:

Apr 26 14:04:45 hostname 565049: Apr 26 18:04:43.476 UTC: %EARL-DFC1-1-EXCESSIVE_PARITY_ERROR: EARL 0: Parity error detected in VRAM'

which parses into:

Testing message: program='EARL-DFC1-1-EXCESSIVE_PARITY_ERROR' message='Apr 26 14:04:45 hostname 565049: Apr 26 18:04:43.476 UTC: %EARL-DFC1-1-EXCESSIVE_PARITY_ERROR: EARL 0: Parity error detected in VRAM'
Match name='.classifier.rule_id', value='09944c71-95eb-4bc0-8575-936931d85715', expected='09944c71-95eb-4bc0-8575-936931d85715'
Wrong match name='.cisco.Facility', value='EARL', expected='EARL-DFC1'
Wrong match name='.cisco.Severity', value='DFC1', expected='1'
Wrong match name='.cisco.MsgName', value='1-EXCESSIVE_PARITY_ERROR', expected='EXCESSIVE_PARITY_ERROR'
Match name='.cisco.MsgText', value='EARL 0: Parity error detected in VRAM', expected='EARL 0: Parity error detected in VRAM'

I’ve tried a pattern that would parse EARL and DFC1 into separate variables, then append them into .cisco.Facility – which didn’t work. Still that leaves the pattern hardcoded to just two hypens. I’ve also tried hard coding EARL-DFC1 into the pattern, then use value to set .cisco.Facility correctly. This approach works but being hardcoded, another Facility with embedded hyphens will fail. Ideally I’d like to have a pattern that matches on the last hyphen before the severity and capture that. I could parse using a regexp and setting $1, $2, $3, and $4 to the appropriate variables but I’m concerned about performance issues – I’m thinking this is a last resort solution.

Any thoughts on how to proceed?

Regards,

Mark Schoonover – KA6WKE
Infrastructure Engineering Manager, Splunk Architect
ENE   : Tools, Instrumentation and Common Services Team
Office: 32.8697° N, 116.9711° W
Phone : 770-261-7934
Email : mark.schoonover at cigna.com<mailto:mark.schoonover at cigna.com>
HPSM Team: ENE NMS Engineering

------------------------------------------------------------------------------
CONFIDENTIALITY NOTICE: If you have received this email in error,
please immediately notify the sender by e-mail at the address shown.
This email transmission may contain confidential information.  This
information is intended only for the use of the individual(s) or entity to
whom it is intended even if addressed incorrectly.  Please delete it from
your files if you are not the intended recipient.  Thank you for your
compliance.  Copyright (c) 2017 Cigna
==============================================================================

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng<https://urldefense.proofpoint.com/v2/url?u=https-3A__lists.balabit.hu_mailman_listinfo_syslog-2Dng&d=DwMFaQ&c=WlnXFIBCT3pmNh_w8hYeLw&r=7hmZ2rDjdhHVbv2tovCmuIwqUlXIJdVXqIASPRODb2U&m=s7SsNLIK6w8eENvA6fSVfIY_hmH_Ao37Vfkb8LMT5go&s=4rnwzztJes-sEVyNI37_YvvyrlHkRLSwjQGprO-fmX0&e=>
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.balabit.com_support_documentation_-3Fproduct-3Dsyslog-2Dng&d=DwMFaQ&c=WlnXFIBCT3pmNh_w8hYeLw&r=7hmZ2rDjdhHVbv2tovCmuIwqUlXIJdVXqIASPRODb2U&m=s7SsNLIK6w8eENvA6fSVfIY_hmH_Ao37Vfkb8LMT5go&s=k_tNOdB-fJ6oAIfsJ6wU1JVQv5cTPockc95Diky6exc&e=>
FAQ: http://www.balabit.com/wiki/syslog-ng-faq<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.balabit.com_wiki_syslog-2Dng-2Dfaq&d=DwMFaQ&c=WlnXFIBCT3pmNh_w8hYeLw&r=7hmZ2rDjdhHVbv2tovCmuIwqUlXIJdVXqIASPRODb2U&m=s7SsNLIK6w8eENvA6fSVfIY_hmH_Ao37Vfkb8LMT5go&s=rNRCDCuEpMjwyl9gta5U2p59odVIqTgghc5xzMfOU_k&e=>

------------------------------------------------------------------------------
CONFIDENTIALITY NOTICE: If you have received this email in error,
please immediately notify the sender by e-mail at the address shown. 
This email transmission may contain confidential information.  This
information is intended only for the use of the individual(s) or entity to
whom it is intended even if addressed incorrectly.  Please delete it from
your files if you are not the intended recipient.  Thank you for your
compliance.  Copyright (c) 2017 Cigna
==============================================================================
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170502/f35a6b06/attachment-0001.html>