[syslog-ng] Hitting g_assert when using sanitize-utf8 enabled!

[James Elstone]

Hi Bazsi,

The version of glib is 2.46.2 on FreeBSD 10.3.

The issue does not occur on the first packet coming through, but when under light load (~100/sec)...

Tried reducing the number of unprintable chars and now only \0x92 exists in the inbound message it falls over on. It is always a message with \0x92 that causes it to fail.

Is there a way to have a filter applies before the message is utf8_sanitised using a regular expression or the like?

What if the assert was removed, what effect would it have?

Many thanks to all!

Kr,

James

On 6 January 2017 12:49:28 GMT+00:00, "Scheidler, Balázs" <balazs.scheidler at balabit.com> wrote:
>Hi,
>
>Attila is right, it would help a lot to see the original log message
>and
>your glib version. That code path uses a performance hack that relies
>on a
>GLib implementation detail. Either the glib behaviour has changed or
>another assumption fails, but just looking at the code I don't know
>what
>might.
>
>-- 
>Bazsi
>
>On Fri, Jan 6, 2017 at 1:41 PM, Szalai, Attila <
>Attila.Szalai at morganstanley.com> wrote:
>
>> Hi James,
>>
>>
>>
>> Checking the source, it means the following:
>>
>>
>>
>> The code allocate a buffer 6 times bigger than the original string
>length
>> to able to hold the escaped form of the utf-8 character.
>>
>>
>>
>> The assert means, that the string, after escaping was not fit into
>this
>> buffer for some reason. Or, to be more precise, the GString
>implementation
>> decided that it should reallocate the string, which usually only
>happen if
>> the string gets too big to fit into its original place. Currently I
>have no
>> recent glib source to check if I’m right.
>>
>>
>>
>> The original string could help a lot to find the root cause.
>>
>>
>>
>> Ps.: the escaping works as replacing the original byte with \xHH, so
>> theoretically it can only grows from 1 byte to 4, which should fit
>into a
>> buffer 6 times bigger than the original size.
>>
>>
>>
>> *From:* syslog-ng [mailto:syslog-ng-bounces at lists.balabit.hu] *On
>Behalf
>> Of *James Elstone
>> *Sent:* Thursday, January 05, 2017 10:35 PM
>> *To:* syslog-ng at lists.balabit.hu
>> *Subject:* [syslog-ng] Hitting g_assert when using sanitize-utf8
>enabled!
>>
>>
>>
>> Hi Balabit et al,
>>
>> When using the sanitize-utf8 flag I am hitting a g_assert in
>> modules/syslogformat/syslog-format.c; what could be causing this?
>>
>> Any advice welcome!!
>>
>> Kr,
>>
>> James
>> --
>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>
>>
>> ------------------------------
>>
>> NOTICE: Morgan Stanley is not acting as a municipal advisor and the
>> opinions or views contained herein are not intended to be, and do not
>> constitute, advice within the meaning of Section 975 of the
>Dodd-Frank Wall
>> Street Reform and Consumer Protection Act. If you have received this
>> communication in error, please destroy all electronic and paper
>copies and
>> notify the sender immediately. Mistransmission is not intended to
>waive
>> confidentiality or privilege. Morgan Stanley reserves the right, to
>the
>> extent permitted under applicable law, to monitor electronic
>> communications. This message is subject to terms available at the
>following
>> link: http://www.morganstanley.com/disclaimers  If you cannot access
>> these links, please notify us by reply message and we will send the
>> contents to you. By communicating with Morgan Stanley you consent to
>the
>> foregoing and to the voice recording of conversations with personnel
>of
>> Morgan Stanley.
>>
>>
>> ____________________________________________________________
>> __________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?
>> product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>>

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170106/20054377/attachment.html>