<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Hi,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
FreeBSD 12.1 changed from the legacy syslog protocol to RFC 5424 format. When you use the system() source for local logs, this is handled automagically. But you can also fix it by hand:
<a href="https://github.com/syslog-ng/syslog-ng/issues/2428">https://github.com/syslog-ng/syslog-ng/issues/2428</a></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Bye,<br>
</div>
<div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="Signature">
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<div>
<div>
<div dir="ltr">Peter Czanik (CzP) <peter.czanik@oneidentity.com><br>
Balabit (a OneIdentity company) / syslog-ng upstream<br>
<a href="https://syslog-ng.com/community/" target="_blank">https://syslog-ng.com/<wbr>community/</a><br>
<a href="https://twitter.com/PCzanik" target="_blank">https://twitter.com/PCzanik</a></div>
</div>
</div>
<br>
</div>
</div>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Sass, Fabian <Fabian.Sass@f-i-ts.de><br>
<b>Sent:</b> Friday, June 5, 2020 08:28<br>
<b>To:</b> syslog-ng@lists.balabit.hu <syslog-ng@lists.balabit.hu><br>
<b>Subject:</b> [syslog-ng] Message and Header are being split incorrectly</font>
<div> </div>
</div>
<style>
<!--
@font-face
{font-family:"Cambria Math"}
@font-face
{font-family:Calibri}
p.x_MsoNormal, li.x_MsoNormal, div.x_MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif}
a:link, span.x_MsoHyperlink
{color:#0563C1;
text-decoration:underline}
a:visited, span.x_MsoHyperlinkFollowed
{color:#954F72;
text-decoration:underline}
span.x_E-MailFormatvorlage17
{font-family:"Arial",sans-serif;
color:windowtext;
font-weight:normal;
font-style:normal}
.x_MsoChpDefault
{font-family:"Calibri",sans-serif}
@page WordSection1
{margin:70.85pt 70.85pt 2.0cm 70.85pt}
div.x_WordSection1
{}
-->
</style>
<div lang="DE" link="#0563C1" vlink="#954F72">
<div style="background-color:#FFEB9C; width:100%; border-style:solid; border-color:#9C6500; border-width:1pt; padding:2pt; font-size:10pt; line-height:12pt; font-family:'Calibri'; color:Black; text-align:left">
<span style="color:#9C6500; font-weight:bold">CAUTION:</span> This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.</div>
<br>
<div>
<div class="x_WordSection1">
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">Hi,<br>
<br>
since updating the Base OS to FreeBSD 12.1 syslog-ng 3.27.1 fails splitting a message and its headers.<br>
<br>
<br>
Jun 5 08:12:00 myhostname 1 2020-06-05T08:12:00.042109+02:00 myhostname /usr/sbin/cron 71149 - - (root) RELOAD (tabs/root)</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">Jun 5 08:12:01 myhostname 1 2020-06-05T08:12:01.546089+02:00 myhostname named 54403 - - client @0xfffffff 0.0.0.0: update 'some.domain/IN' denied<br>
<br>
</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"><br>
Using templates for the destination reveals that the $MSGHDR Macro only holds the value “1”, which is the wrongly extracted $PROGRAM macro.
</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">$MESSAGE itself contains almost all information of the lines above, to be precise everything from (including) the $ISODATE to the end of the line.<br>
However using templates and rewrite rules is in this case sufficient to restore the logformat that was used before the update.</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"><br>
The bigger issue is that changing the value of $PROGRAM has no effect when sending it to antoher syslog-ng loghost.<br>
The behavior seems to be analog to this bug:<br>
<a href="https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fpipermail%2Fsyslog-ng%2F2011-August%2F017132.html&data=02%7C01%7CPeter.Czanik%40oneidentity.com%7C4be342ff90b940df59d008d80919b8e7%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637269353395272488&sdata=h4MEu8GH%2BKK6tbdxi8BvV6r5eBvoZ%2BF44JfzMLj6TbU%3D&reserved=0" originalsrc="https://lists.balabit.hu/pipermail/syslog-ng/2011-August/017132.html" shash="iQoBh8aqun0rGu5Vl3ksMuNIyR7gHkTbkH1GRlJwCSEC+371YUY/5tpKFvAbtI5hjtgMwrp979ksSKuOqsbNxWPx/zuYgk4TVjBPgX5+Qd3ixAcpvVfvJZ5o1IMciOWNZP/QBvwngVuqvP84xesZEfddQddXDmOkpt/10n9XqRw=">https://lists.balabit.hu/pipermail/syslog-ng/2011-August/017132.html</a></span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"><br>
As you can see in my syslog-ng.conf the $PROGRAM macro is overwritten to “named” if named was logging to the local syslog-ng. The successful overwriting is verified using a separate logfile (destination d_test):</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">@version: 3.5</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">#</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"># options</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">#</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">options {</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> mark_freq(3600);</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> flush_lines(0);</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> dir_perm(0640);</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> chain_hostnames(off);</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> keep_hostname(yes);</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> create_dirs(yes);</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> use_dns(yes);</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> dns_cache(yes);</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> dns_cache_expire(3600);</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">};<br>
source s_all { unix-dgram("/var/run/log");</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> unix-dgram("/var/run/logpriv" perm(0600));</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> internal();</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">};</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">#</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"># rewrite since syslog message splitting is broken since update to freebsd12...</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">#</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">rewrite r_msg {</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> set(</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> "named", value("PROGRAM") condition(message(".* named [0-9]+ - -.*"))</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> );</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> subst(".* ([a-zA-Z/\._]+) ([0-9]+) - - (.*)", "$1[$2]: $3", value("MESSAGE"));</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">};</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">#</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"># destinations</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">#</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">destination d_test { file("/var/log/fabian_messages" template("$DATE $PROGRAM $HOST $MESSAGE\n")); };</span></p>
<p class="x_MsoNormal"><span lang="FR" style="font-size:10.0pt; font-family:"Arial",sans-serif">destination d_test2 { file("/var/log/fabian_messages2"); };</span></p>
<p class="x_MsoNormal"><span lang="FR" style="font-size:10.0pt; font-family:"Arial",sans-serif">destination d_messages { file("/var/log/messages" template("$DATE $HOST $MESSAGE\n")); };</span></p>
<p class="x_MsoNormal"><span lang="FR" style="font-size:10.0pt; font-family:"Arial",sans-serif">destination d_loghost { tcp("someiphere" port(514) template("$DATE $HOST $MESSAGE\n")); udp("anotheriphere" port(10525) template("$DATE $HOST $MESSAGE\n")); };</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">#</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">#</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"># logging</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">#</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">log { source(s_all); rewrite(r_msg); destination(d_messages); };</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">log { source(s_all); rewrite(r_msg); destination(d_loghost); };</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">log { source(s_all); rewrite(r_msg); destination(d_test); };</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">log { source(s_all); destination(d_test2); };</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">Am I missing something here, or is syslog-ng somehow behaving unintended?</span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif"> </span></p>
<p class="x_MsoNormal"><span lang="EN-US" style="font-size:10.0pt; font-family:"Arial",sans-serif">Fabian</span></p>
</div>
</div>
</div>
</body>
</html>