<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:"Yu Gothic";
panose-1:2 11 4 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI";
panose-1:2 11 5 2 4 2 4 2 2 3;}
@font-face
{font-family:"\@Yu Gothic";
panose-1:2 11 4 0 0 0 0 0 0 0;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.xmsonormal, li.xmsonormal, div.xmsonormal
{mso-style-name:x_msonormal;
margin:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle21
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal">Hi Gabor, <o:p></o:p></p>
<p class="MsoNormal">Thanks for the reply. Just looking over the Cisco CSL made my head ache, so I’m glad to hear you think it’s complex too.
<o:p></o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">We’re running Cisco <span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;color:#242424;background:white">
NGFW 1140 as the devices sending the data. Probably running Cisco Firepower 6.6.x of some sort, but I don’t have the exact version number handy. I’ll see if I can find it.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;color:#242424;background:white"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;color:#242424;background:white">If I know what I want to search for, is there a way to quickly write a simple SCL to handle just this format? I really need to be able to classify
by the levels, so the FTD-(\d)-(\d+) where the $1 turns into the level number I can filter against to decide which messages to forward on to another destination would be awesome.
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;color:#242424;background:white"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;color:#242424;background:white">Thanks,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.5pt;font-family:"Segoe UI",sans-serif;color:#242424;background:white">John<o:p></o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1F497D">Sr. Storage Architect</span><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;color:red">TOSHIBA AMERICA, INC.<o:p></o:p></span></b></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1F497D">1251 6<sup>th</sup>, Ave 41<sup>st</sup> flr, New York, NY 10020</span><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1F497D">508-736-5499 (mobile)</span><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1F497D">E-Mail: </span>
<u><span style="font-size:10.0pt;color:blue"><a href="mailto:john.stoffel@toshiba.com"><span style="color:#0563C1">john.stoffel@toshiba.com</span></a></span></u><span style="color:#1F497D"><o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:10.0pt;color:#1F497D">Website: </span>
<u><span style="font-size:10.0pt;color:#0563C1"><a href="https://nassc.service-now.com/ess/navpage.do"><span style="color:#0563C1">Service Now Self Service Portal</span></a></span></u><span style="color:#1F497D"><o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b>From:</b> Gabor Nagy (gnagy) <Gabor.Nagy@oneidentity.com>
<br>
<b>Sent:</b> Monday, February 28, 2022 5:26 AM<br>
<b>To:</b> Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu>; Stoffel, John (TAI) <john.stoffel@toshiba.com><br>
<b>Subject:</b> Re: parsing cisco firepower logs problem with 3.33<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="color:black">Dear John!<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><br>
Sorry for not answering earlier.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Thanks for the detailed report of this issue.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">To be honest, cisco-parser is probably the most complex SCL in syslog-ng, and it's hard to debug it.<br>
Message processing can be debugged if syslog-ng is running with trace-level debugging, but it's not an easy output to parse.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black;background:white">The internal logs show what happens to a log message on each pipeline element (from sources until it reaches the destination).</span><span style="color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Trace level internal logs causes vast amount of logs on the console or internal() log, so I recommend using this only for debugging 1 message.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">It can be turned on via "syslog-ng-ctl trace -s 1" or starting syslog-ng in the foreground: "syslog-ng -Fedvt".<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">I've checked the log formats you sent us, and the main problem is not with the order of elements, but the format of the timestamp.<br>
It's an ISO-8601 formatted timestamp, while the cisco-parser only supports the old "day-name month" format (e.g. Feb 16 2022 16:31:53).<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">When I've changed only the timestamp format on one of your log messages, cisco-parser() worked:<br>
<166>Feb 16 2022 16:31:53 na-zy-int-fp1140-p02 : %FTD-6-305012: Teardown dynamic TCP translation from FOO-WAN_IN:10.92.60.80/59877 to FOO-OUTSIDE:6.7.8.18/59877 duration 0:01:01<br>
<br>
Also with the changed order the hostname (or by Cisco terminology "origin-id") cannot be parsed by the cisco-parser.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">I'll create a pull request about this and discuss it with the team.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Can you send us some information about that Cisco device that sends these logs, please? So we can look into it's documentation.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><br>
<br>
Regards,<br>
Gabor<o:p></o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="98%" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> syslog-ng <<a href="mailto:syslog-ng-bounces@lists.balabit.hu">syslog-ng-bounces@lists.balabit.hu</a>> on behalf of Stoffel, John (TAI) <<a href="mailto:John.Stoffel@toshiba.com">John.Stoffel@toshiba.com</a>><br>
<b>Sent:</b> Thursday, February 17, 2022 15:47<br>
<b>To:</b> <a href="mailto:syslog-ng@lists.balabit.hu">syslog-ng@lists.balabit.hu</a> <<a href="mailto:syslog-ng@lists.balabit.hu">syslog-ng@lists.balabit.hu</a>><br>
<b>Subject:</b> [syslog-ng] parsing cisco firepower logs problem with 3.33</span>
<o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div style="border:solid #9C6500 1.0pt;padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="MsoNormal" style="line-height:12.0pt;background:#FFEB9C"><b><span style="font-size:10.0pt;color:#9C6500">CAUTION:</span></b><span style="font-size:10.0pt;color:black"> This email originated from outside of the organization. Do not follow guidance,
click links, or open attachments unless you recognize the sender and know the content is safe.<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Hi,<o:p></o:p></p>
<p class="xmsonormal">I'm trying to parse some cisco logs from a Cisco firepower firewall, using syslog-ng v3.33 on a CentOS 7 system. After pounding my head against the wall a few times to realize that you can't just re-start syslog-ng and have it re-read
a source file from scratch... that instead I need to just push the data using netcat, it's now in a state where I think I can try to debug things.<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">My logs look like this:<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal"><166>2022-02-16T15:31:53Z na-zy-int-fp1140-p02 : %FTD-6-305012: Teardown dynamic UDP translation fr<o:p></o:p></p>
<p class="xmsonormal">om TAI-INSIDE:1.2.3.110/51288 to FOO-OUTSIDE:6.7.8.18/33333 duration 0:00:00<o:p></o:p></p>
<p class="xmsonormal"><166>2022-02-16T15:31:53Z na-zy-int-fp1140-p02 : %FTD-6-305012: Teardown dynamic TCP translation fr<o:p></o:p></p>
<p class="xmsonormal">om FOO-WAN_IN:10.92.60.80/59877 to FOO-OUTSIDE:6.7.8.18/59877 duration 0:01:01<o:p></o:p></p>
<p class="xmsonormal"><166>2022-02-16T15:31:53Z na-zy-int-fp1140-p02 : %FTD-6-305011: Built dynamic UDP translation from<o:p></o:p></p>
<p class="xmsonormal">FOO-INSIDE:1.2.3.110/51288 to FOO-OUTSIDE:6.7.8.18/5632<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Looking at this log, vs the examples given in the /usr/share/syslog-ng/include/scl/cisco/plugin.conf file, I think the problem is that my logs shows the:<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal"> sequence, date: origin, %MSG <o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">instead of <o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal"> sequence, origin, date: %MSG<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">and it’s not clear to me how I would hack the plugin.conf file to handle this issue. My end goal is to be able to parse the message enough by log level so I can forward only a subset of messages to another remote syslog system.
<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal">Thanks,<o:p></o:p></p>
<p class="xmsonormal">John<o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;color:#1F497D"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;color:#1F497D">Sr. Storage Architect</span><o:p></o:p></p>
<p class="xmsonormal"><b><span style="font-size:10.0pt;color:red">TOSHIBA AMERICA, INC.</span></b><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;color:#1F497D">1251 6<sup>th</sup>, Ave 41<sup>st</sup> flr, New York, NY 10020</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;color:#1F497D">508-736-5499 (mobile)</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;color:#1F497D">E-Mail: </span>
<u><span style="font-size:10.0pt;color:blue"><a href="mailto:john.stoffel@toshiba.com"><span style="color:#0563C1">john.stoffel@toshiba.com</span></a></span></u><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:10.0pt;color:#1F497D">Website: </span>
<u><span style="font-size:10.0pt;color:#0563C1"><a href="https://urldefense.com/v3/__https:/nam12.safelinks.protection.outlook.com/?url=https*3A*2F*2Fnassc.service-now.com*2Fess*2Fnavpage.do&data=04*7C01*7Cgabor.nagy*40oneidentity.com*7Ce1fc0e410cf542f2294e08d9f22481a5*7C91c369b51c9e439c989c1867ec606603*7C0*7C1*7C637807060893690199*7CUnknown*7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0*3D*7C3000&sdata=u0eNB5EHzsyTSOvNbI7czRJLxpvC2EPeeKsZ6H5X9q0*3D&reserved=0__;JSUlJSUlJSUlJSUlJSUlJSUl!!BiNunAf9XXY-!R4NbMeGvRLi2JniMHFDJNW1kydS0JyHKyMA48a4Y9i-LYsY-BKG3QcjH71lz5Iw8hNbi$"><span style="color:#0563C1">Service
Now Self Service Portal</span></a></span></u><o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
</div>
</div>
</div>
</body>
</html>