[syslog-ng] parsing cisco firepower logs problem with 3.33

Mon Feb 28 10:26:14 UTC 2022

Dear John!

Sorry for not answering earlier.

Thanks for the detailed report of this issue.

To be honest, cisco-parser is probably the most complex SCL in syslog-ng, and it's hard to debug it.
Message processing can be debugged if syslog-ng is running with trace-level debugging, but it's not an easy output to parse.
The internal logs show what happens to a log message on each pipeline element (from sources until it reaches the destination).
Trace level internal logs causes vast amount of logs on the console or internal() log, so I recommend using this only for debugging 1 message.
It can be turned on via "syslog-ng-ctl trace -s 1" or starting syslog-ng in the foreground: "syslog-ng -Fedvt".

I've checked the log formats you sent us, and the main problem is not with the order of elements, but the format of the timestamp.
It's an ISO-8601 formatted timestamp, while the cisco-parser only supports the old "day-name month" format (e.g. Feb 16 2022 16:31:53).
When I've changed only the timestamp format on one of your log messages, cisco-parser() worked:
<166>Feb 16 2022 16:31:53 na-zy-int-fp1140-p02 : %FTD-6-305012: Teardown dynamic TCP translation from FOO-WAN_IN:10.92.60.80/59877 to FOO-OUTSIDE:6.7.8.18/59877 duration 0:01:01

Also with the changed order the hostname (or by Cisco terminology "origin-id") cannot be parsed by the cisco-parser.

I'll create a pull request about this and discuss it with the team.
Can you send us some information about that Cisco device that sends these logs, please? So we can look into it's documentation.

Regards,
Gabor
________________________________
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of Stoffel, John (TAI) <John.Stoffel at toshiba.com>
Sent: Thursday, February 17, 2022 15:47
To: syslog-ng at lists.balabit.hu <syslog-ng at lists.balabit.hu>
Subject: [syslog-ng] parsing cisco firepower logs problem with 3.33

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.

Hi,

I'm trying to parse some cisco logs from a Cisco firepower firewall, using syslog-ng v3.33 on a CentOS 7 system.  After pounding my head against the wall a few times to realize that you can't just re-start syslog-ng and have it re-read a source file from scratch... that instead I need to just push the data using netcat, it's now in a state where I think I can try to debug things.

My logs look like this:

<166>2022-02-16T15:31:53Z na-zy-int-fp1140-p02 : %FTD-6-305012: Teardown dynamic UDP translation fr

om TAI-INSIDE:1.2.3.110/51288 to FOO-OUTSIDE:6.7.8.18/33333 duration 0:00:00

<166>2022-02-16T15:31:53Z na-zy-int-fp1140-p02 : %FTD-6-305012: Teardown dynamic TCP translation fr

om FOO-WAN_IN:10.92.60.80/59877 to FOO-OUTSIDE:6.7.8.18/59877 duration 0:01:01

<166>2022-02-16T15:31:53Z na-zy-int-fp1140-p02 : %FTD-6-305011: Built dynamic UDP translation from

FOO-INSIDE:1.2.3.110/51288 to FOO-OUTSIDE:6.7.8.18/5632

Looking at this log, vs the examples given in the /usr/share/syslog-ng/include/scl/cisco/plugin.conf file, I think the problem is that my logs shows the:

   sequence, date: origin, %MSG

instead of

  sequence, origin, date: %MSG

and it’s not clear to me how I would hack the plugin.conf file to handle this issue.  My end goal is to be able to parse the message enough by log level so I can forward only a subset of messages to another remote syslog system.

Thanks,

John

Sr. Storage Architect

TOSHIBA AMERICA, INC.

1251 6th,  Ave 41st flr, New York, NY 10020

508-736-5499 (mobile)

E-Mail:  john.stoffel at toshiba.com<mailto:john.stoffel at toshiba.com>

Website: Service Now Self Service Portal<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnassc.service-now.com%2Fess%2Fnavpage.do&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7Ce1fc0e410cf542f2294e08d9f22481a5%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637807060893690199%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=u0eNB5EHzsyTSOvNbI7czRJLxpvC2EPeeKsZ6H5X9q0%3D&reserved=0>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20220228/931e6eb5/attachment.htm>