<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
Hi,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="appendonsend"></div>
<div>
<div>
<div class="x_ydp6e8e0c06yahoo-style-wrap" style="font-family:Helvetica Neue,Helvetica,Arial,sans-serif; font-size:16px">
<div dir="ltr"><span>
<ul dir="ltr" style="color:rgb(0,0,0); font-family:Calibri,Arial,Helvetica,sans-serif; font-size:14.6667px">
<li>How does the so-rcvbuf of the kernel (rmem_max) correlate to the so-rcvbuf in syslog-ng, especially when using so-reuseport. Should the rmem_max be the sum of all the so-rcvbuf settings in syslog-ng? Is rmem_max a per CPU buffer? In general, how should
these variables be scaled in unison? In our case, our so-rcvbuf is <span>441,326,592 across 4 so-reuseports, while our rmem_max is <span>268,435,456.</span></span></li></ul>
<div><span>The so-rcvbuf() option of syslog-ng sets the SO_RCVBUF socket options, so the two are the same (syslog-ng sets the kernel buffer size). When using
<span>so-reuseport</span>(), each UDP source consists of a separate socket. <span>
<span>so-rcvbuf() </span></span>can be set separately for each <span><span></span></span>socket if necessary.<br>
</span></div>
<div><span>Note that the kernel doubles this value (to allow space for bookkeeping overhead).<br>
</span></div>
<div><span>We have a warning in case the value is not accepted by the kernel:</span></div>
<div><i><span> The kernel refused to set the receive buffer (SO_RCVBUF) to the requested size, you probably need to adjust buffer related kernel parameters</span></i></div>
<div><i><span><br>
</span></i></div>
<div><i><span></span></i>net.core.rmem_max specifies the maximum configurable value, but it is per-socket limit, so it does NOT have to be the sum of all sockets.</div>
<div>The default value can be also set with net.core.rmem_default, but <span><span>you can override this with the so-rcvbuf()</span></span> syslog-ng option up to the
<span>rmem_max</span> limit.<br>
</div>
<div><br>
</div>
<ul dir="ltr" style="color:rgb(0,0,0); font-family:Calibri,Arial,Helvetica,sans-serif; font-size:14.6667px">
<li><span><span>We record all of our syslog-ng stats at 5 minute intervals with syslog-ng-ctl. If this never shows any drops, would that imply that this is more of an issue with kernel buffers than syslog-ng itself?</span></span></li></ul>
<div>In case of UDP, you can't avoid message loss, so it does not really matter where messages are dropped. If you configured flags(flow-control) in any of the UDP log paths, syslog-ng would not drop messages, but the kernel will definitely do it instead. I
don't recommend using flow-control for UDP sources. Without flow control, syslog-ng reads messages as fast as it can and drops new messages when a destination becomes unavailable/too slow to process the throughput. This can be monitored with syslog-ng-ctl
(dropped counters).<br>
</div>
<div>Kernel message drops are also possible in this scenario, but it means that the given syslog-ng config/environment is unable to process the incoming traffic fast enough, so something has to be adjusted or scaled.<br>
</div>
<div><br>
</div>
<ul dir="ltr" style="color:rgb(0,0,0); font-family:Calibri,Arial,Helvetica,sans-serif; font-size:14.6667px">
<li><span><span>Is there a syslog-ng command that will show its queues, to see how those are being utilized?</span></span></li></ul>
</span></div>
All destination queues can be monitored with syslog-ng-ctl stats. Those counters end with the "queued" suffix, for example:</div>
<div class="x_ydp6e8e0c06yahoo-style-wrap" style="font-family:Helvetica Neue,Helvetica,Arial,sans-serif; font-size:16px">
dst.network;d_ise#1;tcp,127.0.0.1:5555;a;queued;1</div>
<div class="x_ydp6e8e0c06yahoo-style-wrap" style="font-family:Helvetica Neue,Helvetica,Arial,sans-serif; font-size:16px">
<br>
</div>
<div class="x_ydp6e8e0c06yahoo-style-wrap" style="font-family:Helvetica Neue,Helvetica,Arial,sans-serif; font-size:16px">
--</div>
<div class="x_ydp6e8e0c06yahoo-style-wrap" style="font-family:Helvetica Neue,Helvetica,Arial,sans-serif; font-size:16px">
László Várady<br>
</div>
</div>
</div>
</body>
</html>