[syslog-ng] Problems parsing Cisco syslogs

[Scheidler, Balázs]

The cisco-parser() should take care about messages from cisco routers
assuming they were received using flags(no-parse). It will automatically
detect timestamp formats, but its not perfect, so if you encounter
something that it doesn't properly parse, do let us know, so we can add it.

Also, default-network-drivers() makes it possible to receive both cisco and
non-cisco logs on the same port, automatically recognizing the appropriate
format. This driver relies on the app-parser() framework, which can be
extended by application specific parsers. With that you can construct your
specific source driver configuration if you find default-network-drivers ()
too complicated.

You can find all of these in the documentation, and their source in the
syslog-ng configuration library (scl for short, usually in
/usr/share/syslog-ng/include/scl).

On Tue, Feb 26, 2019, 18:08 Sandor Geller <sandor.geller at ericsson.com wrote:

> Hello,
>
> When the no-parse flag is used then the macros referencing various parts
> of the message aren't filled in. HOST could get looked up using a reverse
> DNS lookup unless the keep_hostname option is set. The syslog priority is
> set to user.notice when parsing is disabled.
>
> Did you take a look at the existing cisco parser? Using it or adopting it
> should ease your job. If you could configure the Ciscos to use other port
> than anything else (which speaks syslog, Cisco devices usually aren't
> such...) would be even better.
>
> Regards,
> Sandor
>
> On 02/26/2019 05:36 PM, N. Max Pierson wrote:
>
> Hi List,
>
> I have been trying to get something in place that can parse syslogs from
> various Cisco devices. The message format is almost the same with a few
> exceptions. Here is what I have tried and it works but now it has created
> another problem I do not know how to troubleshoot.
>
> So that I could see exactly what was being parsed, I disabled the default
> parsing using the below.
>
> source s_network { udp(ip(0.0.0.0) port(514) flags(no-parse)); };
>
> rewrite r_cisco{
> subst('^<\d+>(\d+:|:)\s+(\.\w+|\w+)\s+\d+\s+\d+\s\d+:\d+:\d+:\s|^<\d+>:\s+\d+\s+\w+\s+\d+\s+\d+:\d+:\d+\s\w+:\s|^<\d+>(\d+:|:)\s',
> "", value("MESSAGE"), type("pcre"), flags("ignore-case")); };
>
> destination d_mysql {
>     sql(type(mysql)
>     host("127.0.0.1")
>     username("syslog-ng")
>     password("password")
>     database("syslog")
>     table("messages_${HOST}")
>     columns("datetime datetime", "host varchar(50)", "level varchar(10)",
> "message text")
>     values("${R_YEAR}-${R_MONTH}-${R_DAY} ${R_HOUR}:${R_MIN}:${R_SEC}",
> "${HOST}", "${LEVEL}", "${MESSAGE}")
>     indexes("datetime", "level")
>     );
> };
>
> log { source(s_network); rewrite(r_cisco); destination(d_mysql); };
>
> This works perfectly as it formats the message as I want and covers IOS
> and NX-OS devices. The problem is when I turned off the default parser, now
> all of my logs show "notice" in the $LEVEL macro and doesn't reflect the
> real message header level. The $HOST macro still works fine however.
>
> Is this the expected behavior that the message header fields are not
> parsed as well as the $MESSAGE itself not being parsed? How can map the
> header level field properly to the $LEVEL marco if I disable the default
> parser?
>
> Regards,
> Max
>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20190226/9af618c4/attachment.html>