[syslog-ng] syslog-ng 3.22.1 consuming a lot of RAM on amqp or kafka destinations - patterndb
joao reis
joao at 7lan.net
Thu Oct 24 18:54:31 UTC 2019
Hi,
Sorry for the delay, I've attached the configuration files.
The problem occurs in 3.24 too.
I will run valgrind to try to find out something.
Thanks in advance.
On 8/26/19 4:32 AM, Attila Szakacs (aszakacs) wrote:
> Hi Joao,
>
> Could you please share the whole config file, or the minimal config
> parts for the problem to happen, so we can reproduce the issue
> locally? The patterndb xml file will probably also be needed.
> There were some PRs merged related to patterndb since 3.17, but I
> cannot say for sure, which would introduce a memory leak.
>
> If you would like to debug syslog-ng for yourself, you can run it with
> valgrind.
> For example: valgrind --leak-check=full sbin/syslog-ng -Fedtv
>
> Thanks!
>
> Best regards,
> Attila
> ------------------------------------------------------------------------
> *From:* syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of
> joao reis <joao at 7lan.net>
> *Sent:* Sunday, August 25, 2019 9:19 PM
> *To:* syslog-ng at lists.balabit.hu <syslog-ng at lists.balabit.hu>
> *Subject:* Re: [syslog-ng] syslog-ng 3.22.1 consuming a lot of RAM on
> amqp or kafka destinations - patterndb
> CAUTION: This email originated from outside of the organization. Do
> not follow guidance, click links, or open attachments unless you
> recognize the sender and know the content is safe.
>
>
> Ok, I did another tests and discover that the problem is not amqp or
> kafka destination, but the parserdb.
>
> Removing the parser in the log line, the memory keeps normal.
>
> Downgrading to syslog-ng 3.17 I can use the parserdb with amqp without
> the memory leak problem. I didn't test with newer versions yet, but
> seems to be a memory leak problem with patterndb. I will remove the
> "<action>" sections in patterndb to see if it change something in 3.22...
>
> Any known issue?
>
> On 8/24/19 9:18 AM, joao reis wrote:
> > Hi,
> >
> > I'm having high RAM consuming problems using both amqp or kafka
> > destinations. I'm using patterndb (without any context definition)
> > some basic filters.
> >
> > Changing the destination to "file" or "program" type, the high RAM
> > problem doesn't happen.
> >
> > I tested these backends only in this version of syslog-ng and I don't
> > know if it is a version specific problem.
> >
> > The backends are up and running and receiving correctly from syslog-ng.
> >
> > Here are my configurations:
> >
> > destination d_amqp {
> > amqp(
> > vhost("/email_log")
> > host("127.0.0.1")
> > port(5672)
> > exchange("maillog")
> > username("maillog")
> > password("maillog")
> > disk-buffer(
> > mem-buf-size(10000)
> > disk-buf-size(2000000)
> > reliable(yes)
> > dir("/tmp/disk-buffer")
> > )
> > exchange-type("fanout")
> > routing-key("")
> > persistent(yes)
> > value-pairs(
> > scope("nv-pairs")
> > )
> > );
> > };
> >
> >
> > destination d_kafka {
> > kafka(
> > bootstrap-servers("127.0.0.1:9092")
> > topic("MYTOPIC")
> > );
> > };
> >
> >
> > I added the disk-buffer option to see if it gets any better, but
> > without any good result;
> >
> > Here the syslog-ng-ctl output, when configured to amqp:
> >
> > [root at logserver3 conf.d]# syslog-ng-ctl stats
> > SourceName;SourceId;SourceInstance;State;Type;Number
> > global;payload_reallocs;;a;processed;3635299
> > src.journald;s_sys#0;journal;a;processed;542
> > src.journald;s_sys#0;journal;a;stamp;1566648214
> > global;sdata_updates;;a;processed;0
> > global;scratch_buffers_bytes;;a;queued;512
> > source;s_host;;a;processed;5440633
> > destination;d_boot;;a;processed;0
> > destination;d_kern;;a;processed;9
> > source;s_sys;;a;processed;612
> >
> dst.amqp;d_amqp#0;amqp,/email_log,127.0.0.1,5672,maillog,fanout;a;dropped;0
> >
> >
> dst.amqp;d_amqp#0;amqp,/email_log,127.0.0.1,5672,maillog,fanout;a;processed;2691676
> >
> >
> dst.amqp;d_amqp#0;amqp,/email_log,127.0.0.1,5672,maillog,fanout;a;queued;0
> >
> >
> dst.amqp;d_amqp#0;amqp,/email_log,127.0.0.1,5672,maillog,fanout;a;written;2691676
> >
> > src.internal;s_sys#1;;a;processed;70
> > src.internal;s_sys#1;;a;stamp;1566648339
> > global;msg_clones;;a;processed;2691634
> > global;internal_queue_length;;a;processed;0
> > destination;d_spol;;a;processed;0
> > destination;d_mlal;;a;processed;0
> > center;;received;a;processed;5441245
> > destination;d_mesg;;a;processed;531
> > destination;d_mail;;a;processed;0
> > destination;d_auth;;a;processed;29
> > destination;d_cron;;a;processed;52
> > destination;d_amqp;;a;processed;2691676
> > global;scratch_buffers_count;;a;queued;5970004541496
> > center;;queued;a;processed;2692297
> >
> > Thanks in advance.
> >
> >
> >
> >
> ______________________________________________________________________________
> >
> > Member info:
> https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C3f366968cc6848e50dfc08d7299137f4%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637023576067773885&sdata=IevvrtaCVev4wWIIkX2qQ3USXFqgswZE%2Fr%2BSN7fps7I%3D&reserved=0
> > Documentation:
> >
> https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C3f366968cc6848e50dfc08d7299137f4%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637023576067773885&sdata=o4nPzuf08QzuA0gWM2YhR46tBfzYxCk2rEcAHB%2B2BYM%3D&reserved=0
> > FAQ:
> https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C3f366968cc6848e50dfc08d7299137f4%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637023576067773885&sdata=1UinGzeqP9y8I42N0m2pnTtSYk%2BCzpN7SWuhPhGyhLs%3D&reserved=0
> >
> ______________________________________________________________________________
> Member info:
> https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C3f366968cc6848e50dfc08d7299137f4%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637023576067773885&sdata=IevvrtaCVev4wWIIkX2qQ3USXFqgswZE%2Fr%2BSN7fps7I%3D&reserved=0
> Documentation:
> https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C3f366968cc6848e50dfc08d7299137f4%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637023576067773885&sdata=o4nPzuf08QzuA0gWM2YhR46tBfzYxCk2rEcAHB%2B2BYM%3D&reserved=0
> FAQ:
> https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7CAttila.Szakacs%40oneidentity.com%7C3f366968cc6848e50dfc08d7299137f4%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637023576067773885&sdata=1UinGzeqP9y8I42N0m2pnTtSYk%2BCzpN7SWuhPhGyhLs%3D&reserved=0
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20191024/286dd41d/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: syslog-ng-clean.tar.bz2
Type: application/x-bzip
Size: 8412 bytes
Desc: not available
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20191024/286dd41d/attachment-0001.bin>
More information about the syslog-ng
mailing list