I’ve checked the glib source too (in version 2.50, but I do not think  it changed too much between the two version) and have no idea how this could happen.

 

So, an example line is definitively needed to find the root cause.

 

On the other hand, there is a trick in that code to save a malloc and a “static”[*] buffer is used in that code. Therefore if that buffer is reallocated (and therefore the “static” buffer is freed, that means that the memory gets to be corrupted.

 

[*] Practically the buffer is allocated from the stack, but it’s working just like a static buffer from the malloc point of view. It should not be freed.

 

From: syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of James Elstone
Sent: Friday, January 06, 2017 2:55 PM
To: Syslog-ng users' and developers' mailing list
Subject: Re: [syslog-ng] Hitting g_assert when using sanitize-utf8 enabled!

 

Sorry; update - It happens on the first packet that contains \x092 when sanitize-utf8 is enabled; consistently.

Running glib 2.46.2 with Syslog-ng 3.7.3 on FreeBSD 10.3.

Any ideas please?

Kr,

James

James

On 6 January 2017 13:38:58 GMT+00:00, James Elstone <james@elstone.net> wrote:

Hi Bazsi,

The version of glib is 2.46.2 on FreeBSD 10.3.

The issue does not occur on the first packet coming through, but when under light load (~100/sec)...

Tried reducing the number of unprintable chars and now only \0x92 exists in the inbound message it falls over on. It is always a message with \0x92 that causes it to fail.

Is there a way to have a filter applies before the message is utf8_sanitised using a regular expression or the like?

What if the assert was removed, what effect would it have?

Many thanks to all!

Kr,

James

On 6 January 2017 12:49:28 GMT+00:00, "Scheidler, Balázs" <balazs.scheidler@balabit.com> wrote:

Hi,

Attila is right, it would help a lot to see the original log message and your glib version. That code path uses a performance hack that relies on a GLib implementation detail. Either the glib behaviour has changed or another assumption fails, but just looking at the code I don't know what might.


--
Bazsi

 

On Fri, Jan 6, 2017 at 1:41 PM, Szalai, Attila <Attila.Szalai@morganstanley.com> wrote:

Hi James,

 

Checking the source, it means the following:

 

The code allocate a buffer 6 times bigger than the original string length to able to hold the escaped form of the utf-8 character.

 

The assert means, that the string, after escaping was not fit into this buffer for some reason. Or, to be more precise, the GString implementation decided that it should reallocate the string, which usually only happen if the string gets too big to fit into its original place. Currently I have no recent glib source to check if I’m right.

 

The original string could help a lot to find the root cause.

 

Ps.: the escaping works as replacing the original byte with \xHH, so theoretically it can only grows from 1 byte to 4, which should fit into a buffer 6 times bigger than the original size.

 

From: syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of James Elstone
Sent: Thursday, January 05, 2017 10:35 PM
To: syslog-ng@lists.balabit.hu
Subject: [syslog-ng] Hitting g_assert when using sanitize-utf8 enabled!

 

Hi Balabit et al,

When using the sanitize-utf8 flag I am hitting a g_assert in modules/syslogformat/syslog-format.c; what could be causing this?

Any advice welcome!!

Kr,

James
--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

 


NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.


______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

 

 


--
Sent from my Android device with K-9 Mail. Please excuse my brevity.




NOTICE: Morgan Stanley is not acting as a municipal advisor and the opinions or views contained herein are not intended to be, and do not constitute, advice within the meaning of Section 975 of the Dodd-Frank Wall Street Reform and Consumer Protection Act. If you have received this communication in error, please destroy all electronic and paper copies and notify the sender immediately. Mistransmission is not intended to waive confidentiality or privilege. Morgan Stanley reserves the right, to the extent permitted under applicable law, to monitor electronic communications. This message is subject to terms available at the following link: http://www.morganstanley.com/disclaimers  If you cannot access these links, please notify us by reply message and we will send the contents to you. By communicating with Morgan Stanley you consent to the foregoing and to the voice recording of conversations with personnel of Morgan Stanley.