<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hello Saqib M,</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
I think your problem still can be just a small misconfiguration. It would be really good if you can share the exact version of Syslog-ng, your whole configuration and multiple example messages. So we won't accidentally overlook something. (<span style="color: rgb(0, 0, 0); font-family: Arial, Helvetica, sans-serif; font-size: 12pt;">If
your configuration contains private information you can either scramble it or send it in private to one of us.)</span></div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Example: (What was Gábor referring to) With a fresh Syslog-ng, you should see these additional debug messages in the output:</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<div style="color:#c5c8c6;background-color:#1e1e1e;font-family:'Droid Sans Mono', 'monospace', monospace, 'Droid Sans Fallback';font-weight:normal;font-size:12px;line-height:16px">
<div><span>Incoming log entry; line</span><span style="color:#676867">=</span><span style="color:#9aa83a">'default send string'</span></div>
<div><span>Initial message parsing follows</span><span style="color:#ff0b00">;</span></div>
<div><span>Setting value; name</span><span style="color:#676867">=</span><span style="color:#9aa83a">'PROGRAM'</span><span>, value</span><span style="color:#676867">=</span><span style="color:#9aa83a">'default'</span><span>, msg</span><span style="color:#676867">=</span><span style="color:#9aa83a">'0x7f8214006fc0'</span></div>
<div><span>Setting value; name</span><span style="color:#676867">=</span><span style="color:#9aa83a">'LEGACY_MSGHDR'</span><span>, value</span><span style="color:#676867">=</span><span style="color:#9aa83a">'default '</span><span>, msg</span><span style="color:#676867">=</span><span style="color:#9aa83a">'0x7f8214006fc0'</span></div>
<div><span>Setting value; name</span><span style="color:#676867">=</span><span style="color:#9aa83a">'MESSAGE'</span><span>, value</span><span style="color:#676867">=</span><span style="color:#9aa83a">'send string'</span><span>, msg</span><span style="color:#676867">=</span><span style="color:#9aa83a">'0x7f8214006fc0'</span></div>
</div>
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
>From this you can see, that the "udp" and "network" sources (without any other instruction) are trying to parse the incoming message as an RFC5424 message, which will result to unexpected values in certain macros. For example, in this particular case the "MESSAGE"
macro only contains the string "send string", that is why "default send string" do not match on it.</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Best regards,</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Laci</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Gabor Nagy (gnagy) <Gabor.Nagy@oneidentity.com><br>
<b>Sent:</b> Friday, December 4, 2020 10:29<br>
<b>To:</b> Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu><br>
<b>Subject:</b> Re: [syslog-ng] Filter matching not working</font>
<div> </div>
</div>
<style type="text/css" style="display:none">
<!--
p
{margin-top:0;
margin-bottom:0}
-->
</style>
<div dir="ltr">
<div style="background-color:#FFEB9C; width:100%; border-style:solid; border-color:#9C6500; border-width:1pt; padding:2pt; font-size:10pt; line-height:12pt; font-family:'Calibri'; color:Black; text-align:left">
<span style="color:#9C6500; font-weight:bold">CAUTION:</span> This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.</div>
<br>
<div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
I still can't see more debug logs in your internal log. I guess you are using an older version of syslog-ng. Unfortunately trace level debugging can only be used if syslog-ng was compiled with trace level support.<br>
Can you tell us what version of syslog-ng are you using?</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
<span style="color:rgb(0,0,0); font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt">Can you test your filter with the following BSD format message, please?</span><br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
<13>Dec 4 10:27:20 localhost myprogram: default send string</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:11pt; color:rgb(0,0,0)">
Regards,<br>
Gabor</div>
<div id="x_appendonsend"></div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="x_divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Saqib M <saqib.m@cummins.com><br>
<b>Sent:</b> Thursday, December 3, 2020 16:26<br>
<b>To:</b> Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu><br>
<b>Subject:</b> Re: [syslog-ng] Filter matching not working</font>
<div> </div>
</div>
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
p.x_x_MsoNormal, li.x_x_MsoNormal, div.x_x_MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
a:link, span.x_x_MsoHyperlink
{color:#0563C1;
text-decoration:underline}
a:visited, span.x_x_MsoHyperlinkFollowed
{color:#954F72;
text-decoration:underline}
p.x_x_msonormal0, li.x_x_msonormal0, div.x_x_msonormal0
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
p.x_x_xmsonormal, li.x_x_xmsonormal, div.x_x_xmsonormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
p.x_x_xmsonormal0, li.x_x_xmsonormal0, div.x_x_xmsonormal0
{margin-right:0in;
margin-left:0in;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
p.x_x_xmsochpdefault, li.x_x_xmsochpdefault, div.x_x_xmsochpdefault
{margin-right:0in;
margin-left:0in;
font-size:10.0pt;
font-family:"Calibri",sans-serif}
span.x_x_xmsohyperlink
{color:#0563C1;
text-decoration:underline}
span.x_x_xmsohyperlinkfollowed
{color:#954F72;
text-decoration:underline}
span.x_x_xemailstyle18
{font-family:"Calibri",sans-serif;
color:windowtext}
span.x_x_xemailstyle19
{font-family:"Calibri",sans-serif;
color:windowtext}
span.x_x_EmailStyle27
{font-family:"Calibri",sans-serif;
color:windowtext}
.x_x_MsoChpDefault
{font-size:10.0pt}
@page WordSection1
{margin:1.0in 1.0in 1.0in 1.0in}
-->
</style>
<div lang="EN-US">
<div style="background-color:#FFEB9C; width:100%; border-style:solid; border-color:#9C6500; border-width:1pt; padding:2pt; font-size:10pt; line-height:12pt; font-family:'Calibri'; color:Black; text-align:left">
<span style="color:#9C6500; font-weight:bold">CAUTION:</span> This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.</div>
<br>
<div>
<div class="x_x_WordSection1">
<p class="x_x_MsoNormal">Thank you for getting back on this. I am using the following command but I don’t see the parsing logs.
</p>
<p class="x_x_MsoNormal"> </p>
<p class="x_x_MsoNormal">/opt/syslog-ng/sbin/syslog-ng -e -F -d -v -t</p>
<p class="x_x_MsoNormal"> </p>
<p class="x_x_MsoNormal">This is what I saw.</p>
<p class="x_x_MsoNormal"> </p>
<p class="x_x_MsoNormal">[2020-12-03T15:03:06+0000] Incoming log entry; source='s_net#0', line='default send string'</p>
<p class="x_x_MsoNormal">[2020-12-03T15:03:06+0000] Filter rule evaluation begins; filter_rule='f_discreg'</p>
<p class="x_x_MsoNormal">[2020-12-03T15:03:06+0000] Filter node evaluation result; filter_result='not-match'</p>
<p class="x_x_MsoNormal">[2020-12-03T15:03:06+0000] Filter rule evaluation result; filter_result='not-match', filter_rule='f_discreg'</p>
<p class="x_x_MsoNormal">[2020-12-03T15:03:06+0000] Filter rule evaluation begins; filter_rule='f_dlptracker'</p>
<p class="x_x_MsoNormal">[2020-12-03T15:03:06+0000] Filter node evaluation result; filter_result='not-match'</p>
<p class="x_x_MsoNormal">[2020-12-03T15:03:06+0000] Filter rule evaluation result; filter_result='not-match', filter_rule='f_dlptracker'</p>
<p class="x_x_MsoNormal">[2020-12-03T15:03:06+0000] Outgoing message; destination='d_fallback#0', message='2020-12-03T15:03:06+00:00 172.17.236.3 default send string\x0a'</p>
<p class="x_x_MsoNormal"> </p>
<p class="x_x_MsoNormal">Also, the <span style="color:black">f_discreg is in a log path.
</span></p>
<p class="x_x_MsoNormal">log { source(s_net); filter(f_discreg); destination(d_discard); flags(final); };</p>
<p class="x_x_MsoNormal"> </p>
<p class="x_x_MsoNormal">Could there be other ways to look into it a bit further to see what’s going wrong?</p>
<p class="x_x_MsoNormal"> </p>
<p class="x_x_MsoNormal">Thank you</p>
<p class="x_x_MsoNormal"> </p>
<div>
<p class="x_x_MsoNormal"><span style="font-size:10.0pt">Regards,</span></p>
<p class="x_x_MsoNormal"><span style="font-size:10.0pt"> </span></p>
<p class="x_x_MsoNormal"><span style="font-size:10.0pt">Saqib M</span></p>
<p class="x_x_MsoNormal"><i><span style="font-size:10.0pt">Cybersecurity Co-op</span></i></p>
<p class="x_x_MsoNormal"><span style="font-size:10.0pt">Global Cybersecurity Technologies</span></p>
<p class="x_x_MsoNormal"><span style="font-size:10.0pt">Email: <a href="mailto:saqib.m@cummins.com">
saqib.m@cummins.com</a></span></p>
<p class="x_x_MsoNormal"><span style="font-size:10.0pt">Cummins Inc.</span></p>
</div>
<p class="x_x_MsoNormal"> </p>
<div>
<div style="border:none; border-top:solid #E1E1E1 1.0pt; padding:3.0pt 0in 0in 0in">
<p class="x_x_MsoNormal"><b>From:</b> syslog-ng <syslog-ng-bounces@lists.balabit.hu>
<b>On Behalf Of </b>Gabor Nagy (gnagy)<br>
<b>Sent:</b> Thursday, December 3, 2020 3:17 AM<br>
<b>To:</b> Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu><br>
<b>Subject:</b> Re: [syslog-ng] Filter matching not working</p>
</div>
</div>
<p class="x_x_MsoNormal"> </p>
<p align="right" style="text-align:right"><b><span style="font-size:12.0pt; color:#F39C12">External Sender</span></b></p>
<div>
<div>
<p class="x_x_MsoNormal"><span style="color:black">Hello!<br>
<br>
I just took a quick look on the config and on the internal logs. Couldn't be that "f_discreg" is not included in a log path?<br>
<br>
Another thing that could be is parsing: your "s_net" source will try to parse incoming messages as either BSD or Syslog format.<br>
If you've just tested the filter by sending in the message "default send string", then syslog-ng will parse it, which results that MSG macro will not contain "default send string".</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="color:black">With trace level logging you can debug how does the message being parsed (-t command line option), and to disable parsing on the source side use the flags("no-parse") option.<br>
<br>
See an example:<br>
[2020-12-03T09:14:51.016256] Incoming log entry; line='default send string' </span>
</p>
<div>
<p class="x_x_MsoNormal"><span style="color:black">[2020-12-03T09:14:51.016290] Initial message parsing follows;</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="color:black">[2020-12-03T09:14:51.016322] Setting value; name='PROGRAM', value='default', msg='0x7f24a8005f30'</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="color:black">[2020-12-03T09:14:51.016334] Setting value; name='LEGACY_MSGHDR', value='default ', msg='0x7f24a8005f30'</span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="color:black">[2020-12-03T09:14:51.016341] Setting value; name='MESSAGE', value='send string', msg='0x7f24a8005f30'</span></p>
</div>
<p class="x_x_MsoNormal" style="margin-bottom:12.0pt"><span style="color:black"> </span></p>
</div>
<div>
<p class="x_x_MsoNormal"><span style="color:black"><br>
Regards,<br>
Gabor</span></p>
</div>
<div class="x_x_MsoNormal" align="center" style="text-align:center">
<hr size="2" width="98%" align="center">
</div>
<div id="x_x_divRplyFwdMsg">
<p class="x_x_MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> syslog-ng <<a href="mailto:syslog-ng-bounces@lists.balabit.hu">syslog-ng-bounces@lists.balabit.hu</a>> on behalf of Saqib M <<a href="mailto:saqib.m@cummins.com">saqib.m@cummins.com</a>><br>
<b>Sent:</b> Thursday, December 3, 2020 1:48<br>
<b>To:</b> <a href="mailto:syslog-ng@lists.balabit.hu">syslog-ng@lists.balabit.hu</a> <<a href="mailto:syslog-ng@lists.balabit.hu">syslog-ng@lists.balabit.hu</a>><br>
<b>Subject:</b> [syslog-ng] Filter matching not working</span> </p>
<div>
<p class="x_x_MsoNormal"> </p>
</div>
</div>
<div>
<div style="border:solid #9C6500 1.0pt; padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="x_x_MsoNormal" style="line-height:12.0pt; background:#FFEB9C"><b><span style="font-size:10.0pt; color:#9C6500">CAUTION:</span></b><span style="font-size:10.0pt; color:black"> This email originated from outside of the organization. Do not follow guidance,
click links, or open attachments unless you recognize the sender and know the content is safe.</span></p>
</div>
<p class="x_x_MsoNormal"> </p>
<div>
<div>
<p class="x_x_xmsonormal">Greetings –</p>
<p class="x_x_xmsonormal"> </p>
<p class="x_x_xmsonormal">I have been trying to create a very basic filter that looks up a string in the incoming log. However, it would not match any filter and would go to the default filter. I have tried both match() and message(), neither worked for me.
Please let me know if you think I am missing something.</p>
<p class="x_x_xmsonormal"> </p>
<p class="x_x_xmsonormal"><b>Following are the chunks from the syslog-ng.conf</b></p>
<p class="x_x_xmsonormal"> </p>
<p class="x_x_xmsonormal">source s_net {</p>
<p class="x_x_xmsonormal"># All syslog traffic on port 514 - this is direct from network devices.</p>
<p class="x_x_xmsonormal"> udp(port (514));</p>
<p class="x_x_xmsonormal"> network(transport("tcp") max-connections(20000) log_iw_size(100000000) ); # tags("fortigate", "cisco", "default") );</p>
<p class="x_x_xmsonormal">};</p>
<p class="x_x_xmsonormal"> </p>
<p class="x_x_xmsonormal">filter f_discreg { message("default send string") };</p>
<p class="x_x_xmsonormal"> </p>
<p class="x_x_xmsonormal">log { source(s_net); filter(f_dlptracker); destination(d_dlptracker); flags(final); };</p>
<p class="x_x_xmsonormal"> </p>
<p class="x_x_xmsonormal"><b>Here is the log from the test I ran.</b></p>
<p class="x_x_xmsonormal"><b> </b></p>
<p class="x_x_xmsonormal">[2020-12-02T22:00:36+0000] Incoming log entry; source='s_net#0', line='default send string'</p>
<p class="x_x_xmsonormal">[2020-12-02T22:00:46+0000] Filter node evaluation result; filter_result='not-match', filter_type='OR'</p>
<p class="x_x_xmsonormal">[2020-12-02T22:00:46+0000] Filter rule evaluation result; filter_result='not-match', filter_rule='f_tanium'</p>
<p class="x_x_xmsonormal">[2020-12-02T22:00:46+0000] Filter rule evaluation begins; filter_rule='f_palo_alto'</p>
<p class="x_x_xmsonormal">[2020-12-02T22:00:46+0000] Filter node evaluation result; filter_result='not-match'</p>
<p class="x_x_xmsonormal">[2020-12-02T22:00:46+0000] Filter node evaluation result; filter_result='not-match'</p>
<p class="x_x_xmsonormal">[2020-12-02T22:00:46+0000] Filter node evaluation result; filter_result='not-match'</p>
<p class="x_x_xmsonormal">[2020-12-02T22:00:46+0000] Filter node evaluation result; filter_result='not-match', filter_type='OR'</p>
<p class="x_x_xmsonormal">[2020-12-02T22:00:46+0000] Filter node evaluation result; filter_result='not-match', filter_type='OR'</p>
<p class="x_x_xmsonormal">[2020-12-02T22:00:46+0000] Filter rule evaluation result; filter_result='not-match', filter_rule='f_palo_alto'</p>
<p class="x_x_xmsonormal">[2020-12-02T22:00:46+0000] Filter rule evaluation begins; filter_rule='f_dlptracker'</p>
<p class="x_x_xmsonormal">[2020-12-02T22:00:46+0000] Filter node evaluation result; filter_result='not-match'</p>
<p class="x_x_xmsonormal">[2020-12-02T22:00:46+0000] Filter rule evaluation result; filter_result='not-match', filter_rule='f_dlptracker'</p>
<p class="x_x_xmsonormal">[2020-12-02T22:00:46+0000] Outgoing message; destination='d_fallback#0', message='2020-12-02T22:00:46+00:00 172.17.236.3 default send string\x0a'</p>
<p class="x_x_xmsonormal"> </p>
<p class="x_x_xmsonormal"><span style="font-size:10.0pt">Regards,</span></p>
<p class="x_x_xmsonormal"><span style="font-size:10.0pt"> </span></p>
<p class="x_x_xmsonormal"><span style="font-size:10.0pt">Saqib M</span></p>
<p class="x_x_xmsonormal"><i><span style="font-size:10.0pt">Cybersecurity Co-op</span></i></p>
<p class="x_x_xmsonormal"><span style="font-size:10.0pt">Global Cybersecurity Technologies</span></p>
<p class="x_x_xmsonormal"><span style="font-size:10.0pt">Email: <a href="mailto:saqib.m@cummins.com">
saqib.m@cummins.com</a></span></p>
<p class="x_x_xmsonormal"><span style="font-size:10.0pt">Cummins Inc.</span></p>
<p class="x_x_xmsonormal"> </p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>