[syslog-ng] Hitting g_assert when using sanitize-utf8 enabled!

[James Elstone]

Sorry; update - It happens on the first packet that contains \x092 when sanitize-utf8 is enabled; consistently.

Running glib 2.46.2 with Syslog-ng 3.7.3 on FreeBSD 10.3. 

Any ideas please?

Kr,

James 

James

On 6 January 2017 13:38:58 GMT+00:00, James Elstone <james at elstone.net> wrote:
>Hi Bazsi,
>
>The version of glib is 2.46.2 on FreeBSD 10.3.
>
>The issue does not occur on the first packet coming through, but when
>under light load (~100/sec)...
>
>Tried reducing the number of unprintable chars and now only \0x92
>exists in the inbound message it falls over on. It is always a message
>with \0x92 that causes it to fail.
>
>Is there a way to have a filter applies before the message is
>utf8_sanitised using a regular expression or the like?
>
>What if the assert was removed, what effect would it have?
>
>Many thanks to all!
>
>Kr,
>
>James
>
>On 6 January 2017 12:49:28 GMT+00:00, "Scheidler, Balázs"
><balazs.scheidler at balabit.com> wrote:
>>Hi,
>>
>>Attila is right, it would help a lot to see the original log message
>>and
>>your glib version. That code path uses a performance hack that relies
>>on a
>>GLib implementation detail. Either the glib behaviour has changed or
>>another assumption fails, but just looking at the code I don't know
>>what
>>might.
>>
>>-- 
>>Bazsi
>>
>>On Fri, Jan 6, 2017 at 1:41 PM, Szalai, Attila <
>>Attila.Szalai at morganstanley.com> wrote:
>>
>>> Hi James,
>>>
>>>
>>>
>>> Checking the source, it means the following:
>>>
>>>
>>>
>>> The code allocate a buffer 6 times bigger than the original string
>>length
>>> to able to hold the escaped form of the utf-8 character.
>>>
>>>
>>>
>>> The assert means, that the string, after escaping was not fit into
>>this
>>> buffer for some reason. Or, to be more precise, the GString
>>implementation
>>> decided that it should reallocate the string, which usually only
>>happen if
>>> the string gets too big to fit into its original place. Currently I
>>have no
>>> recent glib source to check if I’m right.
>>>
>>>
>>>
>>> The original string could help a lot to find the root cause.
>>>
>>>
>>>
>>> Ps.: the escaping works as replacing the original byte with \xHH, so
>>> theoretically it can only grows from 1 byte to 4, which should fit
>>into a
>>> buffer 6 times bigger than the original size.
>>>
>>>
>>>
>>> *From:* syslog-ng [mailto:syslog-ng-bounces at lists.balabit.hu] *On
>>Behalf
>>> Of *James Elstone
>>> *Sent:* Thursday, January 05, 2017 10:35 PM
>>> *To:* syslog-ng at lists.balabit.hu
>>> *Subject:* [syslog-ng] Hitting g_assert when using sanitize-utf8
>>enabled!
>>>
>>>
>>>
>>> Hi Balabit et al,
>>>
>>> When using the sanitize-utf8 flag I am hitting a g_assert in
>>> modules/syslogformat/syslog-format.c; what could be causing this?
>>>
>>> Any advice welcome!!
>>>
>>> Kr,
>>>
>>> James
>>> --
>>> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>>>
>>>
>>> ------------------------------
>>>
>>> NOTICE: Morgan Stanley is not acting as a municipal advisor and the
>>> opinions or views contained herein are not intended to be, and do
>not
>>> constitute, advice within the meaning of Section 975 of the
>>Dodd-Frank Wall
>>> Street Reform and Consumer Protection Act. If you have received this
>>> communication in error, please destroy all electronic and paper
>>copies and
>>> notify the sender immediately. Mistransmission is not intended to
>>waive
>>> confidentiality or privilege. Morgan Stanley reserves the right, to
>>the
>>> extent permitted under applicable law, to monitor electronic
>>> communications. This message is subject to terms available at the
>>following
>>> link: http://www.morganstanley.com/disclaimers  If you cannot access
>>> these links, please notify us by reply message and we will send the
>>> contents to you. By communicating with Morgan Stanley you consent to
>>the
>>> foregoing and to the voice recording of conversations with personnel
>>of
>>> Morgan Stanley.
>>>
>>>
>>> ____________________________________________________________
>>> __________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation: http://www.balabit.com/support/documentation/?
>>> product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>>
>>>
>
>-- 
>Sent from my Android device with K-9 Mail. Please excuse my brevity.

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170106/ea9847fc/attachment-0001.html>