[syslog-ng] Syslog-ng shipping logs through AMQP with huge memory leaks

[Nagy, Gábor]

I'm resending this letter to the mail list as we hit the message size
limit. :)

Hello Michal!

I have some good news.
We have found the reason for the increasing memory usage of syslog-ng in
amqp() driver.
It is due to our internal memory handling method (aka scratch_buffers)
which we use in performance critical paths to prevent the overhead of
memory allocations/deallocations of GStrings.
It is using memory from a pre-allocated pool instead of allocating/freeing.

As I have described before, the memory usage only increases if the number
of queued messages reaches a new maximum.
It is not per message and does not happen every for every queued message.

We have several options how to fix this issue and will soon share one.


Regards,
Gabor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180328/c06a3008/attachment.html>