[syslog-ng] Problems parsing Cisco syslogs

[N. Max Pierson]

The cisco-parser works for almost all IOS messages but does not work for
NX-OS messages. I also will need to test IOS-XR messages to see if there's
any variance there as I have those in my network as well, just haven't had
a chance to setup syslog yet. I will give the scl library a good look at
and see if I can make sense of it but I think you have given me enough info
to try and start that way.

thanks,
max

On Tue, Feb 26, 2019 at 11:14 AM Scheidler, Balázs <
balazs.scheidler at oneidentity.com> wrote:

> The cisco-parser() should take care about messages from cisco routers
> assuming they were received using flags(no-parse). It will automatically
> detect timestamp formats, but its not perfect, so if you encounter
> something that it doesn't properly parse, do let us know, so we can add it.
>
> Also, default-network-drivers() makes it possible to receive both cisco
> and non-cisco logs on the same port, automatically recognizing the
> appropriate format. This driver relies on the app-parser() framework, which
> can be extended by application specific parsers. With that you can
> construct your specific source driver configuration if you find
> default-network-drivers () too complicated.
>
> You can find all of these in the documentation, and their source in the
> syslog-ng configuration library (scl for short, usually in
> /usr/share/syslog-ng/include/scl).
>
> On Tue, Feb 26, 2019, 18:08 Sandor Geller <sandor.geller at ericsson.com
> wrote:
>
>> Hello,
>>
>> When the no-parse flag is used then the macros referencing various parts
>> of the message aren't filled in. HOST could get looked up using a reverse
>> DNS lookup unless the keep_hostname option is set. The syslog priority is
>> set to user.notice when parsing is disabled.
>>
>> Did you take a look at the existing cisco parser? Using it or adopting it
>> should ease your job. If you could configure the Ciscos to use other port
>> than anything else (which speaks syslog, Cisco devices usually aren't
>> such...) would be even better.
>>
>> Regards,
>> Sandor
>>
>> On 02/26/2019 05:36 PM, N. Max Pierson wrote:
>>
>> Hi List,
>>
>> I have been trying to get something in place that can parse syslogs from
>> various Cisco devices. The message format is almost the same with a few
>> exceptions. Here is what I have tried and it works but now it has created
>> another problem I do not know how to troubleshoot.
>>
>> So that I could see exactly what was being parsed, I disabled the default
>> parsing using the below.
>>
>> source s_network { udp(ip(0.0.0.0) port(514) flags(no-parse)); };
>>
>> rewrite r_cisco{
>> subst('^<\d+>(\d+:|:)\s+(\.\w+|\w+)\s+\d+\s+\d+\s\d+:\d+:\d+:\s|^<\d+>:\s+\d+\s+\w+\s+\d+\s+\d+:\d+:\d+\s\w+:\s|^<\d+>(\d+:|:)\s',
>> "", value("MESSAGE"), type("pcre"), flags("ignore-case")); };
>>
>> destination d_mysql {
>>     sql(type(mysql)
>>     host("127.0.0.1")
>>     username("syslog-ng")
>>     password("password")
>>     database("syslog")
>>     table("messages_${HOST}")
>>     columns("datetime datetime", "host varchar(50)", "level varchar(10)",
>> "message text")
>>     values("${R_YEAR}-${R_MONTH}-${R_DAY} ${R_HOUR}:${R_MIN}:${R_SEC}",
>> "${HOST}", "${LEVEL}", "${MESSAGE}")
>>     indexes("datetime", "level")
>>     );
>> };
>>
>> log { source(s_network); rewrite(r_cisco); destination(d_mysql); };
>>
>> This works perfectly as it formats the message as I want and covers IOS
>> and NX-OS devices. The problem is when I turned off the default parser, now
>> all of my logs show "notice" in the $LEVEL macro and doesn't reflect the
>> real message header level. The $HOST macro still works fine however.
>>
>> Is this the expected behavior that the message header fields are not
>> parsed as well as the $MESSAGE itself not being parsed? How can map the
>> header level field properly to the $LEVEL marco if I disable the default
>> parser?
>>
>> Regards,
>> Max
>>
>>
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20190226/1db56223/attachment-0001.html>