<html>
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8">
</head>
<body text="#000000" bgcolor="#FFFFFF">
We're seeing an issue on our systems where syslog-ng is consuming an
excessive amount of memory, approximately 380mb (RSS) right now on
one of our hosts.<br>
I'm not sure what's causing this as our config is pretty minimal,
and there are no destinations unavailable which would cause it to
buffer. We do have `log-msg-size(65536)` and `log-fifo-size(10000)`
which might have caused it to buffer a large amount some time in the
past, but would it not release these buffers?<br>
<br>
Version 3.13.2 on CentOS/7 (RPM from
<a class="moz-txt-link-freetext" href="https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng313/">https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng313/</a>)<br>
<br>
Config:<br>
<font color="#990000"><tt>@version:3.13</tt><tt><br>
</tt><tt>@include "scl.conf"</tt><tt><br>
</tt><tt><br>
</tt><tt># This configuration is meant to utilize systemd journald</tt><tt><br>
</tt><tt>options {</tt><tt><br>
</tt><tt> flush_lines(0);</tt><tt><br>
</tt><tt> time_reopen(10);</tt><tt><br>
</tt><tt> log-msg-size(65536);</tt><tt><br>
</tt><tt> log_fifo_size(10000);</tt><tt><br>
</tt><tt> chain_hostnames(off);</tt><tt><br>
</tt><tt> use_dns(no);</tt><tt><br>
</tt><tt> use_fqdn(yes);</tt><tt><br>
</tt><tt> keep_hostname(yes);</tt><tt><br>
</tt><tt> create_dirs(no);</tt><tt><br>
</tt><tt> stats_freq(3600);</tt><tt><br>
</tt><tt>};</tt><tt><br>
</tt><tt><br>
</tt><tt>source s_system { system(); };</tt><tt><br>
</tt><tt>source s_internal { internal(); };</tt><tt><br>
</tt><tt><br>
</tt><tt>destination d_journald {</tt><tt><br>
</tt><tt> unix-dgram("/dev/log" flush-lines(1)
template("${MSG}\n"));</tt><tt><br>
</tt><tt>};</tt><tt><br>
</tt><tt><br>
</tt><tt>log { source(s_internal); destination(d_journald); };</tt><tt><br>
</tt><tt><br>
</tt><tt>destination d_loghost {</tt><tt><br>
</tt><tt> syslog("syslog.ecom.chewy.com"</tt><tt><br>
</tt><tt> frac-digits(3)</tt><tt><br>
</tt><tt> log-fifo-size(10000)</tt><tt><br>
</tt><tt> );</tt><tt><br>
</tt><tt>};</tt><tt><br>
</tt><tt><br>
</tt><tt>log { source(s_system); destination(d_loghost); };</tt></font><br>
<br>
Stats:<br>
<br>
<font color="#990000"><tt># syslog-ng-ctl stats</tt><tt><br>
</tt><tt>SourceName;SourceId;SourceInstance;State;Type;Number</tt><tt><br>
</tt><tt>dst.syslog;d_loghost#0;tcp,syslog.ecom.chewy.com:601;a;dropped;163025</tt><tt><br>
</tt><tt>dst.syslog;d_loghost#0;tcp,syslog.ecom.chewy.com:601;a;processed;489030242</tt><tt><br>
</tt><tt>dst.syslog;d_loghost#0;tcp,syslog.ecom.chewy.com:601;a;queued;1</tt><tt><br>
</tt><tt>dst.syslog;d_loghost#0;tcp,syslog.ecom.chewy.com:601;a;written;488867216</tt><tt><br>
</tt><tt>destination;d_journald;;a;processed;1997</tt><tt><br>
</tt><tt>src.internal;s_internal#0;;a;processed;1997</tt><tt><br>
</tt><tt>src.internal;s_internal#0;;a;stamp;1515709115</tt><tt><br>
</tt><tt>center;;received;a;processed;1997</tt><tt><br>
</tt><tt>destination;d_loghost;;a;processed;489030242</tt><tt><br>
</tt><tt>source;s_internal;;a;processed;1997</tt><tt><br>
</tt><tt>center;;queued;a;processed;489032239</tt><tt><br>
</tt><tt>global;scratch_buffers_count;;a;queued;25769803782</tt><tt><br>
</tt><tt>dst.unix-dgram;d_journald#0;unix-dgram,localhost.afunix:/dev/log;a;dropped;0</tt><tt><br>
</tt><tt>dst.unix-dgram;d_journald#0;unix-dgram,localhost.afunix:/dev/log;a;processed;845</tt><tt><br>
</tt><tt>dst.unix-dgram;d_journald#0;unix-dgram,localhost.afunix:/dev/log;a;queued;0</tt><tt><br>
</tt><tt>dst.unix-dgram;d_journald#0;unix-dgram,localhost.afunix:/dev/log;a;written;845</tt><tt><br>
</tt><tt>global;payload_reallocs;;a;processed;2932894697</tt><tt><br>
</tt><tt>src.journald;;journal;a;processed;489030242</tt><tt><br>
</tt><tt>src.journald;;journal;a;stamp;1515710081</tt><tt><br>
</tt><tt>global;sdata_updates;;a;processed;0</tt><tt><br>
</tt><tt>global;scratch_buffers_bytes;;a;queued;0</tt><tt><br>
</tt><tt>global;msg_clones;;a;processed;135</tt><tt><br>
</tt><tt>global;internal_queue_length;;a;processed;0</tt></font><br>
<br>
Thanks<br>
<br>
-Patrick<br>
<br>
<br>
</body>
</html>