[syslog-ng] Syslog-ng shipping logs through AMQP with huge memory leaks

[Michal Purzynski]

Awesome! Do you want me to test it? I guess I can fetch the PR and deploy
the RPM in place of the affected package we have now.

On Wed, Mar 28, 2018 at 9:04 AM, Scheidler, Balázs <
balazs.scheidler at balabit.com> wrote:

> There's a potential fix here: https://github.com/balabit/
> syslog-ng/pull/1946
>
>
> --
> Bazsi
>
> On Wed, Mar 28, 2018 at 10:29 AM, Nagy, Gábor <gabor.nagy at balabit.com>
> wrote:
>
>> I'm resending this letter to the mail list as we hit the message size
>> limit. :)
>>
>> Hello Michal!
>>
>> I have some good news.
>> We have found the reason for the increasing memory usage of syslog-ng in
>> amqp() driver.
>> It is due to our internal memory handling method (aka scratch_buffers)
>> which we use in performance critical paths to prevent the overhead of
>> memory allocations/deallocations of GStrings.
>> It is using memory from a pre-allocated pool instead of
>> allocating/freeing.
>>
>> As I have described before, the memory usage only increases if the number
>> of queued messages reaches a new maximum.
>> It is not per message and does not happen every for every queued message.
>>
>> We have several options how to fix this issue and will soon share one.
>>
>>
>> Regards,
>> Gabor
>>
>>
>> ____________________________________________________________
>> __________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=
>> syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
>>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180328/f1314733/attachment-0001.html>