<HTML xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<HEAD><!-- Template generated by Exclaimer Template Editor on 02:44:18 Saturday, 7 January 2017 -->
<STYLE type=text/css>P.81e0224e-54b4-4bc8-83df-0e9daa05848f {
MARGIN: 0cm 0cm 0pt
}
LI.81e0224e-54b4-4bc8-83df-0e9daa05848f {
MARGIN: 0cm 0cm 0pt
}
DIV.81e0224e-54b4-4bc8-83df-0e9daa05848f {
MARGIN: 0cm 0cm 0pt
}
TABLE.81e0224e-54b4-4bc8-83df-0e9daa05848fTable {
MARGIN: 0cm 0cm 0pt
}
DIV.Section1 {
page: Section1
}
</STYLE>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="Generator" content="Microsoft Word 14 (filtered medium)" />
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
mso-margin-top-alt:auto;
margin-right:0in;
mso-margin-bottom-alt:auto;
margin-left:0in;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Balloon Text Char";
margin:0in;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.BalloonTextChar
{mso-style-name:"Balloon Text Char";
mso-style-priority:99;
mso-style-link:"Balloon Text";
font-family:"Tahoma","sans-serif";}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</HEAD>
<BODY lang="EN-US" link="blue" vlink="purple">
<P>
<div class="WordSection1">
<p class="MsoNormal"><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D>I’ve checked the glib source too (in version 2.50, but I do not think it changed too much between the two version) and have no idea how this could happen.<o:p></o:p></span></p>
<p class="MsoNormal"><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D><o:p> </o:p></span></p>
<p class="MsoNormal"><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D>So, an example line is definitively needed to find the root cause.<o:p></o:p></span></p>
<p class="MsoNormal"><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D><o:p> </o:p></span></p>
<p class="MsoNormal"><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D>On the other hand, there is a trick in that code to save a malloc and a “static”[*] buffer is used in that code. Therefore if that buffer is reallocated (and
therefore the “static” buffer is freed, that means that the memory gets to be corrupted.<o:p></o:p></span></p>
<p class="MsoNormal"><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D><o:p> </o:p></span></p>
<p class="MsoNormal"><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D>[*] Practically the buffer is allocated from the stack, but it’s working just like a static buffer from the malloc point of view. It should not be freed.<o:p></o:p></span></p>
<p class="MsoNormal"><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D><o:p> </o:p></span></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style=font-size:10.0pt;font-family:"Tahoma","sans-serif">From:</span></b><span style=font-size:10.0pt;font-family:"Tahoma","sans-serif"> syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu]
<b>On Behalf Of </b>James Elstone<br />
<b>Sent:</b> Friday, January 06, 2017 2:55 PM<br />
<b>To:</b> Syslog-ng users' and developers' mailing list<br />
<b>Subject:</b> Re: [syslog-ng] Hitting g_assert when using sanitize-utf8 enabled!<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal" style=margin-bottom:12.0pt>Sorry; update - It happens on the first packet that contains \x092 when sanitize-utf8 is enabled; consistently.<br />
<br />
Running glib 2.46.2 with Syslog-ng 3.7.3 on FreeBSD 10.3. <br />
<br />
Any ideas please?<br />
<br />
Kr,<br />
<br />
James <br />
<br />
James<o:p></o:p></p>
<div>
<p class="MsoNormal">On 6 January 2017 13:38:58 GMT+00:00, James Elstone <<a href="mailto:james@elstone.net">james@elstone.net</a>> wrote:<o:p></o:p></p>
<p class="MsoNormal" style=margin-bottom:12.0pt>Hi Bazsi,<br />
<br />
The version of glib is 2.46.2 on FreeBSD 10.3.<br />
<br />
The issue does not occur on the first packet coming through, but when under light load (~100/sec)...<br />
<br />
Tried reducing the number of unprintable chars and now only \0x92 exists in the inbound message it falls over on. It is always a message with \0x92 that causes it to fail.<br />
<br />
Is there a way to have a filter applies before the message is utf8_sanitised using a regular expression or the like?<br />
<br />
What if the assert was removed, what effect would it have?<br />
<br />
Many thanks to all!<br />
<br />
Kr,<br />
<br />
James<o:p></o:p></p>
<div>
<p class="MsoNormal">On 6 January 2017 12:49:28 GMT+00:00, "Scheidler, Balázs" <<a href="mailto:balazs.scheidler@balabit.com">balazs.scheidler@balabit.com</a>> wrote:<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style=margin-bottom:12.0pt>Hi,<o:p></o:p></p>
</div>
<p class="MsoNormal">Attila is right, it would help a lot to see the original log message and your glib version. That code path uses a performance hack that relies on a GLib implementation detail. Either the glib behaviour has changed or another assumption
fails, but just looking at the code I don't know what might.<o:p></o:p></p>
</div>
<div>
<p class="MsoNormal"><br clear="all" />
<o:p></o:p></p>
<div>
<div>
<div>
<p class="MsoNormal">-- <br />
Bazsi<o:p></o:p></p>
</div>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal">On Fri, Jan 6, 2017 at 1:41 PM, Szalai, Attila <<a href="mailto:Attila.Szalai@morganstanley.com" target="_blank">Attila.Szalai@morganstanley.com</a>> wrote:<o:p></o:p></p>
<div>
<div>
<p class="MsoNormal" style=mso-margin-top-alt:auto;mso-margin-bottom-alt:auto><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D>Hi James,</span><o:p></o:p></p>
<p class="MsoNormal" style=mso-margin-top-alt:auto;mso-margin-bottom-alt:auto><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D> </span><o:p></o:p></p>
<p class="MsoNormal" style=mso-margin-top-alt:auto;mso-margin-bottom-alt:auto><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D>Checking the source, it means the following:</span><o:p></o:p></p>
<p class="MsoNormal" style=mso-margin-top-alt:auto;mso-margin-bottom-alt:auto><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D> </span><o:p></o:p></p>
<p class="MsoNormal" style=mso-margin-top-alt:auto;mso-margin-bottom-alt:auto><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D>The code allocate a buffer 6 times bigger than the original string length to able to hold the escaped
form of the utf-8 character.</span><o:p></o:p></p>
<p class="MsoNormal" style=mso-margin-top-alt:auto;mso-margin-bottom-alt:auto><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D> </span><o:p></o:p></p>
<p class="MsoNormal" style=mso-margin-top-alt:auto;mso-margin-bottom-alt:auto><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D>The assert means, that the string, after escaping was not fit into this buffer for some reason. Or,
to be more precise, the GString implementation decided that it should reallocate the string, which usually only happen if the string gets too big to fit into its original place. Currently I have no recent glib source to check if I’m right.</span><o:p></o:p></p>
<p class="MsoNormal" style=mso-margin-top-alt:auto;mso-margin-bottom-alt:auto><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D> </span><o:p></o:p></p>
<p class="MsoNormal" style=mso-margin-top-alt:auto;mso-margin-bottom-alt:auto><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D>The original string could help a lot to find the root cause.</span><o:p></o:p></p>
<p class="MsoNormal" style=mso-margin-top-alt:auto;mso-margin-bottom-alt:auto><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D> </span><o:p></o:p></p>
<p class="MsoNormal" style=mso-margin-top-alt:auto;mso-margin-bottom-alt:auto><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D>Ps.: the escaping works as replacing the original byte with \xHH, so theoretically it can only grows
from 1 byte to 4, which should fit into a buffer 6 times bigger than the original size.</span><o:p></o:p></p>
<p class="MsoNormal" style=mso-margin-top-alt:auto;mso-margin-bottom-alt:auto><span style=font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D> </span><o:p></o:p></p>
<div style="border:none;border-left:solid blue 1.5pt;padding:0in 0in 0in 4.0pt">
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal" style=mso-margin-top-alt:auto;mso-margin-bottom-alt:auto><b><span style=font-size:10.0pt;font-family:"Tahoma","sans-serif">From:</span></b><span style=font-size:10.0pt;font-family:"Tahoma","sans-serif"> syslog-ng [mailto:<a href="mailto:syslog-ng-bounces@lists.balabit.hu" target="_blank">syslog-ng-bounces@lists.balabit.hu</a>]
<b>On Behalf Of </b>James Elstone<br />
<b>Sent:</b> Thursday, January 05, 2017 10:35 PM<br />
<b>To:</b> <a href="mailto:syslog-ng@lists.balabit.hu" target="_blank">syslog-ng@lists.balabit.hu</a><br />
<b>Subject:</b> [syslog-ng] Hitting g_assert when using sanitize-utf8 enabled!</span><o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal" style=mso-margin-top-alt:auto;mso-margin-bottom-alt:auto> <o:p></o:p></p>
<p class="MsoNormal" style=mso-margin-top-alt:auto;mso-margin-bottom-alt:auto>Hi Balabit et al,<br />
<br />
When using the sanitize-utf8 flag I am hitting a g_assert in modules/syslogformat/syslog-format.c; what could be causing this?<br />
<br />
Any advice welcome!!<br />
<br />
Kr,<br />
<br />
James<br />
-- <br />
Sent from my Android device with K-9 Mail. Please excuse my brevity.<o:p></o:p></p>
</div>
</div>
</div>
</div>
<p class="MsoNormal" style=margin-bottom:12.0pt><o:p> </o:p></p>
<div class="MsoNormal" align="center" style=text-align:center>
<hr size="2" width="100%" align="center" id="m_1970712568857058622HR1" />
</div>
<p class="MsoNormal"><br />
<span style=font-size:7.5pt;font-family:"Arial","sans-serif";color:gray>NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section
975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality
or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link:
<a href="http://www.morganstanley.com/disclaimers" target="_blank">http://www.morganstanley.com/disclaimers</a> If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you
consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.</span><o:p></o:p></p>
</div>
<p class="MsoNormal" style=margin-bottom:12.0pt><br />
______________________________________________________________________________<br />
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">
https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br />
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">
http://www.balabit.com/support/documentation/?product=syslog-ng</a><br />
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br />
<br />
<o:p></o:p></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
</div>
<p class="MsoNormal"><br />
-- <br />
Sent from my Android device with K-9 Mail. Please excuse my brevity.<o:p></o:p></p>
</div>
</div>
<BR /><BR />
<HR id=HR1 />
<BR /><SPAN style="FONT-SIZE: 7.5pt; FONT-FAMILY: Arial; COLOR: #808080">NOTICE:
Morgan Stanley is not acting as a municipal advisor and the opinions or views
contained herein are not intended to be, and do not constitute, advice within
the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer
Protection Act. If you have received this communication in error, please destroy
all electronic and paper copies and notify the sender immediately.
Mistransmission is not intended to waive confidentiality or privilege. Morgan
Stanley reserves the right, to the extent permitted under applicable law, to
monitor electronic communications. This message is subject to terms available at
the following link: http://www.morganstanley.com/disclaimers If you cannot
access these links, please notify us by reply message and we will send the
contents to you. By communicating with Morgan Stanley you consent to the
foregoing and to the voice recording of conversations with personnel of Morgan
Stanley.</SPAN><BR />
<P></P>
<P></P>
<P></P>
<P></P></P></BODY>
</HTML>