<html>
<head>
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
</head>
<body>
<div class="moz-cite-prefix">On 5/28/20 7:46 AM, László Várady
(lvarady) wrote:<br>
</div>
<blockquote type="cite"
cite="mid:CH2PR19MB3685DE2C09AFC7E4E8DBFA48F78E0@CH2PR19MB3685.namprd19.prod.outlook.com">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
<div style="font-size:8pt; color:#f58442 ; font-family:
sans-serif; font-style:normal; font-weight:bold; padding:.2em">
Notice: This message was sent from outside the University of
Victoria email system. Please be cautious with links and
sensitive information.
</div>
<br>
<div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
Hi,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<font size="2"><span style="font-size:11pt">> 1. The OS UDP
buffer seems to be 128MB in size and the so_rcvbuf
configured ins 64M in size. Is that because the syslog-ng
configuration of so_rcvbuf is in characters but the OS
buffer is in bytes?</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<font size="2"><span style="font-size:11pt"><br>
</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<font size="2"><span style="font-size:11pt">This is because
the kernel doubles the value set by syslog-ng (to allow
space for bookkeeping overhead), and this doubled value is
returned by getsockopt(2) and other tools.</span></font><br>
</div>
</div>
</blockquote>
<p><br>
</p>
<p>Thanks.<br>
</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:CH2PR19MB3685DE2C09AFC7E4E8DBFA48F78E0@CH2PR19MB3685.namprd19.prod.outlook.com">
<div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<font size="2"><span style="font-size:11pt">> 3. Increasing
the log_iw_size or the log_iw_size actually seems to make
things worse.</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<font size="2"><span style="font-size:11pt"><br>
</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<font size="2"><span style="font-size:11pt">These 2 values </span></font><span
lang="en"><span title="">already seem high enough.</span></span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<span lang="en"><span title="">Disabling flow-control is also
a </span></span>good idea IMO, when using UDP sources.<br>
</div>
</div>
</blockquote>
<p><br>
</p>
<p>Flow control is completely disabled. Or more precisely
unspecified. It occurs to me that perhaps the default is enabled?</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:CH2PR19MB3685DE2C09AFC7E4E8DBFA48F78E0@CH2PR19MB3685.namprd19.prod.outlook.com">
<div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<font size="2"><span style="font-size:11pt">> All
suggestions that help me understand this and help to
minimize the drops are welcome.</span></font></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<font size="2"><span style="font-size:11pt"></span></font><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
Could you share how incoming packets are distributed across
the 8 sockets?</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
The default SO_REUSEPORT mechanism distributes packets based
on the hash of (peer IP address, port) and (local IP address,
port),</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
Hashing collision is also likely to happen [1], so if you
encounter this problem, there are other possible resolutions.
The commercial syslog-ng version has, for example, an<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
udp-balancer() driver, that uses custom BPF programs to
achieve an <span lang="en">
<span title="">even distribution</span></span> of packets.<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
</div>
</blockquote>
<p><br>
</p>
<p>It is a single device, single IP, single port, so there is just
one socket that is overwhelmed. This will probably be the same
failure to process fast enough if we move to a TCP transport.<br>
</p>
<p><br>
</p>
<blockquote type="cite"
cite="mid:CH2PR19MB3685DE2C09AFC7E4E8DBFA48F78E0@CH2PR19MB3685.namprd19.prod.outlook.com">
<div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
[1] <a
href="https://blog.cloudflare.com/how-to-receive-a-million-packets/"
id="LPlnk601598" moz-do-not-send="true">
https://blog.cloudflare.com/how-to-receive-a-million-packets/</a></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif;
font-size: 11pt; color: rgb(0, 0, 0);">
<br>
</div>
</div>
</blockquote>
<p><br>
</p>
<p>I will read through this blog.</p>
<p><br>
</p>
<pre class="moz-signature" cols="72">--
Evan Rempel</pre>
</body>
</html>