<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
span.EmailStyle18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">One must remember that packet loss using UDP is normal. To check to see if you are experiencing any lost packets issue this command:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">netstat -su<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Look at the value for ‘receive buffer errors’. This is a cumulative count of lost UDP packets. Normally, this number should be zero. If the number is greater
than zero, then you are losing UDP packets.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I have found that adding this to your syslog-ng config file on the ‘source’ statement is helpful:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">so-reuseport(1) so_rcvbuf(80000000)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Also, increasing these values in ‘sysctl’ is also helpful<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">net.core.rmem_max
<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">net.core.rmem_default<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">net.ipv4.udp_mem<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Bryan Klimek<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Mayo Clinic<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu]
<b>On Behalf Of </b>William Luiz Ribeiro Vasconcelos Da Silva<br>
<b>Sent:</b> Friday, November 08, 2019 10:47 AM<br>
<b>To:</b> Syslog-ng users' and developers' mailing list<br>
<b>Subject:</b> [EXTERNAL] Re: [syslog-ng] Syslog-ng don't Listening UDP packets PORT(514) - Problem History<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">Hello Again,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">The anwser of command is this:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">netstat -anu |grep 514<br>
udp 0 0 10.46.105.34:514 0.0.0.0:* <o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">Only one line as expected.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">When I try to start the syslog-ng using the command, I see the following error:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">"[2019-11-08T13:37:32.326014] WARNING: Your configuration file uses an obsoleted keyword, please update your configuration; keyword='use_rcptid', change='This has been deprecated,
try use_uniqid() instead', location='/etc/syslog-ng/syslog-ng.conf:24:4'<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">[2019-11-08T13:37:32.326240] Module loaded and initialized successfully; module='system-source'<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">[2019-11-08T13:37:32.326505] Module loaded and initialized successfully; module='sdjournal'<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">[2019-11-08T13:37:32.327037] Module loaded and initialized successfully; module='kvformat'<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">[2019-11-08T13:37:32.327073] Finishing include; content='block parser iptables-parser() at /usr/share/syslog-ng/include/scl/iptables/iptables.conf:23', depth='3'<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">[2019-11-08T13:37:32.327479] Module loaded and initialized successfully; module='csvparser'<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">[2019-11-08T13:37:32.327591] Finishing include; content='block parser sudo-parser() at /usr/share/syslog-ng/include/scl/sudo/sudo.conf:23', depth='3'<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">[2019-11-08T13:37:32.327669] Finishing include; content='parser generator app-parser', depth='2'<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">[2019-11-08T13:37:32.327709] Finishing include; content='source generator system', depth='1'<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">[2019-11-08T13:37:32.328122] Module loaded and initialized successfully; module='afsocket'<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">[2019-11-08T13:37:32.328504] Module loaded and initialized successfully; module='affile'<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">[2019-11-08T13:37:32.329762] Processing @include statement; filename='/etc/syslog-ng/conf.d/*.conf', include-path='/etc/syslog-ng:/usr/share/syslog-ng/include'<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><b><span style="font-family:"Calibri","sans-serif";color:black">[2019-11-08T13:37:32.329836] Error creating persistent state file; filename='/var/lib/syslog-ng/syslog-ng.persist-', error='Permission denied (13)'</span></b><span style="font-family:"Calibri","sans-serif";color:black">"<o:p></o:p></span></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">Could this be the reason for not capturing UDP packets?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">Tks,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">William Luiz<o:p></o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="3" width="98%" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">De:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> syslog-ng <syslog-ng-bounces@lists.balabit.hu> em nome de Zoltan
Pallagi (zpallagi) <Zoltan.Pallagi@oneidentity.com><br>
<b>Enviado:</b> sexta-feira, 8 de novembro de 2019 13:31<br>
<b>Para:</b> Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu><br>
<b>Assunto:</b> Re: [syslog-ng] Syslog-ng don't Listening UDP packets PORT(514) - Problem History</span>
<o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">Hi,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">The most likely reason that another application also listens on this IP:port pair (it is possible in case of UDP and in that case the packet will be captured by the app started
later)<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">Check the output of
<b>netstat -anu |grep 514</b> when syslog-ng does not capture the udp packets. You should see only one line here.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">In other case, you should check the debug log of syslog-ng (syslog-ng -Fevd) perhaps there is some useful information.<o:p></o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="3" width="98%" align="center">
</div>
<div id="x_divRplyFwdMsg">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">Feladó:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> syslog-ng <syslog-ng-bounces@lists.balabit.hu>, meghatalmazó:
William Luiz Ribeiro Vasconcelos Da Silva <wsilva_ericsson@timbrasil.com.br><br>
<b>Elküldve:</b> 2019. november 8., péntek 16:28<br>
<b>Címzett:</b> syslog-ng@lists.balabit.hu <syslog-ng@lists.balabit.hu><br>
<b>Tárgy:</b> [syslog-ng] Syslog-ng don't Listening UDP packets PORT(514) - Problem History</span>
<o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div style="border:solid #9C6500 1.0pt;padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="MsoNormal" style="line-height:12.0pt;background:#FFEB9C"><b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#9C6500">CAUTION:</span></b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:black"> This email
originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black">Good Morning,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"> Yesterday, we performed the first test of syslog-ng 3.24 OSE on a RHEL 7.6 VM in our customer's production environment.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"> And we identified a very strange behavior.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"> We saw through tcpdump that packets were being received by the network card but were not captured by syslog-ng.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"> We have a UDP packet send simulator, and when I was pointing my simulator to another IP on the same machine(<span style="background:white">the users'
connection IP)</span>, syslog-ng does the capping and consequently the treatment we put into the syslog-ng.conf file.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"> What kind of TROUBLESHOOTING we can do in order to identify the reason for syslog-ng to capture packets on the users' connection IP, but for the application
connection IP, it doesn't identify anything.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"> It is informed by the client at the beginning of our project that syslog should always capture all UDP packets received by the application IP.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"> This is the configuration of the "source" we have on this machine in our client's productive environment.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black">source s_model {<o:p></o:p></span></p>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"> #udp(port(514));<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"> udp(ip(10.46.105.34) port(514));<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"> # udp();<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"> #network(transport("udp"));<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"> #network(<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"> # ip("10.46.105.34")<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"> # transport("udp")<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"> #);<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black">};<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="background:white"><span style="font-family:"Calibri","sans-serif";color:black">Tks,<o:p></o:p></span></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><br>
<span style="font-size:7.5pt">Esta mensagem, incluindo seus anexos, pode conter informações privilegiadas e/ou de caráter confidencial, não podendo ser retransmitida sem autorização do remetente. Se você não é o destinatário ou pessoa autorizada para recebê-la,
informamos que o seu uso, divulgação, cópia ou arquivamento são proibidos. Portanto, se você recebeu esta mensagem por engano, por favor nos informe respondendo imediatamente a este e-mail e delete o seu conteúdo.
<o:p></o:p></span></p>
<p><span style="font-size:7.5pt">This message, including its attachments, may contain privileged or confidential information, and it must not be fowarded without the express authorization of the sender. If you are not the intended recipient, we hereby inform
you that the use, disclosure, copy or filing are forbidden. So, if you received this message as a mistake, please inform us by answering this e-mail and deleting its contents
<o:p></o:p></span></p>
<p><span style="font-size:7.5pt">Questo messaggio, inclusi gli allegati, potrebbe contenere informazioni privilegiate e/o riservate, e non deve essere ritrasmesse senza l'autorizzazione del mittente. Se non siete il destinatario o la persona autorizzata a riceverlo,
informiamo che il suo utilizzo, diffusione, copia o archiviazione sono proibite. Quindi, se avete ricevuto questo messaggio per errore, per cortesia ci informi rispondendo immediatamente a questa email e cancelli il suo contenuto
</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</body>
</html>