<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
Dear John!<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
<br>
Sorry for not answering earlier.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
Thanks for the detailed report of this issue.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
To be honest, cisco-parser is probably the most complex SCL in syslog-ng, and it's hard to debug it.<br>
Message processing can be debugged if syslog-ng is running with trace-level debugging, but it's not an easy output to parse.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
<span style="background-color:rgb(255,255,255); display:inline!important"><span>The internal logs </span>show what happens to a log message on each pipeline element (from sources until it reaches the destination).</span><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
<span style="color:rgb(0,0,0); font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt">Trace level internal logs causes vast amount of logs on the console or internal() log</span><span style="color:rgb(0,0,0); font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt">,
so I recommend using this only for debugging 1 message.</span><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
It can be turned on via "syslog-ng-ctl trace -s 1" or starting syslog-ng in the foreground: "syslog-ng -Fedvt".</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
I've checked the log formats you sent us, and the main problem is not with the order of elements, but the format of the timestamp.<br>
It's an ISO-8601 formatted timestamp, while the cisco-parser only supports the old "day-name month" format (e.g. Feb 16 2022 16:31:53).</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
When I've changed only the timestamp format on one of your log messages, cisco-parser() worked:<br>
<166>Feb 16 2022 16:31:53 na-zy-int-fp1140-p02 : %FTD-6-305012: Teardown dynamic TCP translation from FOO-WAN_IN:10.92.60.80/59877 to FOO-OUTSIDE:6.7.8.18/59877 duration 0:01:01<br>
<br>
Also with the changed order the hostname (or by Cisco terminology "origin-id") cannot be parsed by the cisco-parser.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
I'll create a pull request about this and discuss it with the team.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
Can you send us some information about that Cisco device that sends these logs, please? So we can look into it's documentation.</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
<br>
<br>
Regards,<br>
Gabor</div>
<div id="appendonsend"></div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Stoffel, John (TAI) <John.Stoffel@toshiba.com><br>
<b>Sent:</b> Thursday, February 17, 2022 15:47<br>
<b>To:</b> syslog-ng@lists.balabit.hu <syslog-ng@lists.balabit.hu><br>
<b>Subject:</b> [syslog-ng] parsing cisco firepower logs problem with 3.33</font>
<div> </div>
</div>
<div lang="EN-US" style="word-wrap:break-word">
<div style="background-color:#FFEB9C; width:100%; border-style:solid; border-color:#9C6500; border-width:1pt; padding:2pt; font-size:10pt; line-height:12pt; font-family:'Calibri'; color:Black; text-align:left">
<span style="color:#9C6500; font-weight:bold">CAUTION:</span> This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.</div>
<br>
<div>
<div class="x_WordSection1">
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
Hi,</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
I'm trying to parse some cisco logs from a Cisco firepower firewall, using syslog-ng v3.33 on a CentOS 7 system. After pounding my head against the wall a few times to realize that you can't just re-start syslog-ng and have it re-read a source file from scratch...
that instead I need to just push the data using netcat, it's now in a state where I think I can try to debug things.</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
My logs look like this:</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
<166>2022-02-16T15:31:53Z na-zy-int-fp1140-p02 : %FTD-6-305012: Teardown dynamic UDP translation fr</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
om TAI-INSIDE:1.2.3.110/51288 to FOO-OUTSIDE:6.7.8.18/33333 duration 0:00:00</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
<166>2022-02-16T15:31:53Z na-zy-int-fp1140-p02 : %FTD-6-305012: Teardown dynamic TCP translation fr</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
om FOO-WAN_IN:10.92.60.80/59877 to FOO-OUTSIDE:6.7.8.18/59877 duration 0:01:01</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
<166>2022-02-16T15:31:53Z na-zy-int-fp1140-p02 : %FTD-6-305011: Built dynamic UDP translation from</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
FOO-INSIDE:1.2.3.110/51288 to FOO-OUTSIDE:6.7.8.18/5632</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
Looking at this log, vs the examples given in the /usr/share/syslog-ng/include/scl/cisco/plugin.conf file, I think the problem is that my logs shows the:</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
sequence, date: origin, %MSG </p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
instead of </p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
sequence, origin, date: %MSG</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
and it’s not clear to me how I would hack the plugin.conf file to handle this issue. My end goal is to be able to parse the message enough by log level so I can forward only a subset of messages to another remote syslog system.
</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
Thanks,</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
John</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
</p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
<span style="font-size:10.0pt; color:#1F497D"> </span></p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
<span style="font-size:10.0pt; color:#1F497D">Sr. Storage Architect</span><span style="color:#1F497D"></span></p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
<b><span style="font-size:10.0pt; color:red">TOSHIBA AMERICA, INC.</span></b></p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
<span style="font-size:10.0pt; color:#1F497D">1251 6<sup>th</sup>, Ave 41<sup>st</sup> flr, New York, NY 10020</span><span style="color:#1F497D"></span></p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
<span style="font-size:10.0pt; color:#1F497D">508-736-5499 (mobile)</span><span style="color:#1F497D"></span></p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
<span style="font-size:10.0pt; color:#1F497D">E-Mail: </span><u><span style="font-size:10.0pt; color:blue"><a href="mailto:john.stoffel@toshiba.com"><span style="color:#0563C1">john.stoffel@toshiba.com</span></a></span></u><span style="color:#1F497D"></span></p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
<span style="font-size:10.0pt; color:#1F497D">Website: </span><u><span style="font-size:10.0pt; color:#0563C1"><a href="https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fnassc.service-now.com%2Fess%2Fnavpage.do&data=04%7C01%7Cgabor.nagy%40oneidentity.com%7Ce1fc0e410cf542f2294e08d9f22481a5%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637807060893690199%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=u0eNB5EHzsyTSOvNbI7czRJLxpvC2EPeeKsZ6H5X9q0%3D&reserved=0" originalsrc="https://nassc.service-now.com/ess/navpage.do" shash="gU4yNYCRNCdmWrQ3E939TCGQt5wrauGtyBnSVlOfajDMRfoFtbZXEbmPi9T4GkwrUKngkPgAlrVwM6XVBPbsBzdsvXi595pc2CbHvryaW2NA8R9WiPeGGOt57uzsnR9bgORkrH37HoOBJau+eM7NyLxkZYH6L4rb19c1SoCVWRE="><span style="color:#0563C1">Service
Now Self Service Portal</span></a></span></u><span style="color:#1F497D"></span></p>
<p class="x_MsoNormal" style="margin-top: 0px; margin-bottom: 0px;margin-top:0px; margin-bottom:0px; margin-top:0px; margin-bottom:0px; margin:0in; font-size:11pt; font-family:Calibri,sans-serif">
</p>
</div>
</div>
</div>
</body>
</html>