[syslog-ng] Problems parsing Cisco syslogs

Wed Feb 27 17:11:42 UTC 2019

You can use our docs as a guide if it helps:
http://demo.logzilla.net/help/receiving_data/cisco_ios_configuration

We have also encountered issues with some Cisco gear when the show-timezone command is not used (hosts will come in without a hostname, just a : instead)

For example:

On some Cisco IOS versions, it is imperative that this portion of the command be included. Without it, the syslog daemon may detect your device's hostname as a : instead of the actual hostname.
For example:
Hostname Missing
0    :    189    UTC    %SYS-5-CONFIG_I: Configured from console by user1 on vty3 (192.168.2.207)
Correct Hostname
0    192.168.2.252    189    UTC    %SYS-5-CONFIG_I: Configured from console by user1 on vty3 (192.168.2.207)

From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of "N. Pierson" <nmaxpierson at gmail.com>
Reply-To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu>
Date: Tuesday, February 26, 2019 at 12:11 PM
To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu>
Subject: Re: [syslog-ng] Problems parsing Cisco syslogs

I have all devices configured with this exact command minus the show-timezone option and it doesn't seem to make a difference.

On Tue, Feb 26, 2019 at 11:03 AM Nik Ambrosch <nik at ambrosch.com<mailto:nik at ambrosch.com>> wrote:
I had a similar issue, could you try installing this logging configuration on your cisco devices?

service timestamps log datetime msec localtime show-timezone

On Tue, Feb 26, 2019 at 11:36 AM N. Max Pierson <nmaxpierson at gmail.com<mailto:nmaxpierson at gmail.com>> wrote:
Hi List,

I have been trying to get something in place that can parse syslogs from various Cisco devices. The message format is almost the same with a few exceptions. Here is what I have tried and it works but now it has created another problem I do not know how to troubleshoot.

So that I could see exactly what was being parsed, I disabled the default parsing using the below.

source s_network { udp(ip(0.0.0.0) port(514) flags(no-parse)); };

rewrite r_cisco{ subst('^<\d+>(\d+:|:)\s+(\.\w+|\w+)\s+\d+\s+\d+\s\d+:\d+:\d+:\s|^<\d+>:\s+\d+\s+\w+\s+\d+\s+\d+:\d+:\d+\s\w+:\s|^<\d+>(\d+:|:)\s', "", value("MESSAGE"), type("pcre"), flags("ignore-case")); };

destination d_mysql {
    sql(type(mysql)
    host("127.0.0.1")
    username("syslog-ng")
    password("password")
    database("syslog")
    table("messages_${HOST}")
    columns("datetime datetime", "host varchar(50)", "level varchar(10)", "message text")
    values("${R_YEAR}-${R_MONTH}-${R_DAY} ${R_HOUR}:${R_MIN}:${R_SEC}", "${HOST}", "${LEVEL}", "${MESSAGE}")
    indexes("datetime", "level")
    );
};

log { source(s_network); rewrite(r_cisco); destination(d_mysql); };

This works perfectly as it formats the message as I want and covers IOS and NX-OS devices. The problem is when I turned off the default parser, now all of my logs show "notice" in the $LEVEL macro and doesn't reflect the real message header level. The $HOST macro still works fine however.

Is this the expected behavior that the message header fields are not parsed as well as the $MESSAGE itself not being parsed? How can map the header level field properly to the $LEVEL marco if I disable the default parser?

Regards,
Max

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7Ccdukes%40logzilla.net%7C76fb8f1c40154b05489608d69c0d6e63%7C17fac5c255634489bef45cda2e65588f%7C0%7C0%7C636867978833373551&sdata=kulONTiXvj8%2BpodoTWbmdbvS0q5hfJjLm%2FQqmTDbe48%3D&reserved=0>
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7Ccdukes%40logzilla.net%7C76fb8f1c40154b05489608d69c0d6e63%7C17fac5c255634489bef45cda2e65588f%7C0%7C0%7C636867978833383559&sdata=V5DC31aH0fSKIXrguQLBt0iMjAxqMbSx4hZ%2BNhZu3vI%3D&reserved=0>
FAQ: http://www.balabit.com/wiki/syslog-ng-faq<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7Ccdukes%40logzilla.net%7C76fb8f1c40154b05489608d69c0d6e63%7C17fac5c255634489bef45cda2e65588f%7C0%7C0%7C636867978833393568&sdata=iynptNkkmdR%2FAknYSBcoWeUlFAGJuCeZxqpLlDUIVH8%3D&reserved=0>
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng<https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7Ccdukes%40logzilla.net%7C76fb8f1c40154b05489608d69c0d6e63%7C17fac5c255634489bef45cda2e65588f%7C0%7C0%7C636867978833393568&sdata=qvXCBbfURy%2FlNIjCLO7RNz%2BiBhMOxv4GADc6zM1VJys%3D&reserved=0>
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7Ccdukes%40logzilla.net%7C76fb8f1c40154b05489608d69c0d6e63%7C17fac5c255634489bef45cda2e65588f%7C0%7C0%7C636867978833403576&sdata=J8iBIrYuSYmdUlMalObCFx97hKpKRCGjlRtVkvDKEg8%3D&reserved=0>
FAQ: http://www.balabit.com/wiki/syslog-ng-faq<https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7Ccdukes%40logzilla.net%7C76fb8f1c40154b05489608d69c0d6e63%7C17fac5c255634489bef45cda2e65588f%7C0%7C0%7C636867978833403576&sdata=0Zfirx7WqC%2Bbzfaa3GAXR6gplKkP77J4T%2FwYE%2FWh1rg%3D&reserved=0>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20190227/e043ef00/attachment-0001.html>