[syslog-ng] Excessive memory usage

[Scheidler, Balázs]

Hi,

we do release those buffers, but the libc malloc() may not.

It could make sense to LD_PRELOAD the jemalloc allocator, that is better at
returning pages to the system.


-- 
Bazsi

On Thu, Jan 11, 2018 at 11:54 PM, Patrick Hemmer <syslogng at stormcloud9.net>
wrote:

> We're seeing an issue on our systems where syslog-ng is consuming an
> excessive amount of memory, approximately 380mb (RSS) right now on one of
> our hosts.
> I'm not sure what's causing this as our config is pretty minimal, and
> there are no destinations unavailable which would cause it to buffer. We do
> have `log-msg-size(65536)` and `log-fifo-size(10000)` which might have
> caused it to buffer a large amount some time in the past, but would it not
> release these buffers?
>
> Version 3.13.2 on CentOS/7 (RPM from https://copr.fedorainfracloud.
> org/coprs/czanik/syslog-ng313/)
>
> Config:
> @version:3.13
> @include "scl.conf"
>
> # This configuration is meant to utilize systemd journald
> options {
>   flush_lines(0);
>   time_reopen(10);
>   log-msg-size(65536);
>   log_fifo_size(10000);
>   chain_hostnames(off);
>   use_dns(no);
>   use_fqdn(yes);
>   keep_hostname(yes);
>   create_dirs(no);
>   stats_freq(3600);
> };
>
> source s_system { system(); };
> source s_internal { internal(); };
>
> destination d_journald {
>   unix-dgram("/dev/log" flush-lines(1) template("${MSG}\n"));
> };
>
> log { source(s_internal); destination(d_journald); };
>
> destination d_loghost {
>     syslog("syslog.ecom.chewy.com"
>         frac-digits(3)
>         log-fifo-size(10000)
>     );
> };
>
> log { source(s_system); destination(d_loghost); };
>
> Stats:
>
> # syslog-ng-ctl stats
> SourceName;SourceId;SourceInstance;State;Type;Number
> dst.syslog;d_loghost#0;tcp,syslog.ecom.chewy.com:601;a;dropped;163025
> dst.syslog;d_loghost#0;tcp,syslog.ecom.chewy.com:601;a;processed;489030242
> dst.syslog;d_loghost#0;tcp,syslog.ecom.chewy.com:601;a;queued;1
> dst.syslog;d_loghost#0;tcp,syslog.ecom.chewy.com:601;a;written;488867216
> destination;d_journald;;a;processed;1997
> src.internal;s_internal#0;;a;processed;1997
> src.internal;s_internal#0;;a;stamp;1515709115
> center;;received;a;processed;1997
> destination;d_loghost;;a;processed;489030242
> source;s_internal;;a;processed;1997
> center;;queued;a;processed;489032239
> global;scratch_buffers_count;;a;queued;25769803782
> dst.unix-dgram;d_journald#0;unix-dgram,localhost.afunix:/
> dev/log;a;dropped;0
> dst.unix-dgram;d_journald#0;unix-dgram,localhost.afunix:/
> dev/log;a;processed;845
> dst.unix-dgram;d_journald#0;unix-dgram,localhost.afunix:/
> dev/log;a;queued;0
> dst.unix-dgram;d_journald#0;unix-dgram,localhost.afunix:/
> dev/log;a;written;845
> global;payload_reallocs;;a;processed;2932894697
> src.journald;;journal;a;processed;489030242
> src.journald;;journal;a;stamp;1515710081
> global;sdata_updates;;a;processed;0
> global;scratch_buffers_bytes;;a;queued;0
> global;msg_clones;;a;processed;135
> global;internal_queue_length;;a;processed;0
>
> Thanks
>
> -Patrick
>
>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180116/24afd55a/attachment.html>