<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<meta name="Generator" content="Microsoft Word 14 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Verdana;
panose-1:2 11 6 4 3 5 4 4 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
@font-face
{font-family:"Segoe UI Symbol";
panose-1:2 11 5 2 4 2 4 2 2 3;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p
{mso-style-priority:99;
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.xmsonormal, li.xmsonormal, div.xmsonormal
{mso-style-name:x_msonormal;
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.xxmsonormal, li.xxmsonormal, div.xxmsonormal
{mso-style-name:x_xmsonormal;
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
p.xxxmsonormal, li.xxxmsonormal, div.xxxmsonormal
{mso-style-name:x_xxmsonormal;
margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri","sans-serif";}
p.xxxmsochpdefault, li.xxxmsochpdefault, div.xxxmsochpdefault
{mso-style-name:x_xxmsochpdefault;
margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Calibri","sans-serif";}
p.xxmsochpdefault, li.xxmsochpdefault, div.xxmsochpdefault
{mso-style-name:x_xmsochpdefault;
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Times New Roman","serif";}
p.xmsochpdefault, li.xmsochpdefault, div.xmsochpdefault
{mso-style-name:x_msochpdefault;
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:"Times New Roman","serif";}
span.xmsohyperlink
{mso-style-name:x_msohyperlink;
color:blue;
text-decoration:underline;}
span.xmsohyperlinkfollowed
{mso-style-name:x_msohyperlinkfollowed;
color:purple;
text-decoration:underline;}
span.xxmsohyperlink
{mso-style-name:x_xmsohyperlink;
color:blue;
text-decoration:underline;}
span.xxmsohyperlinkfollowed
{mso-style-name:x_xmsohyperlinkfollowed;
color:purple;
text-decoration:underline;}
span.xxxmsohyperlink
{mso-style-name:x_xxmsohyperlink;
color:blue;
text-decoration:underline;}
span.xxxmsohyperlinkfollowed
{mso-style-name:x_xxmsohyperlinkfollowed;
color:purple;
text-decoration:underline;}
span.xxxemailstyle17
{mso-style-name:x_xxemailstyle17;
font-family:"Calibri","sans-serif";
color:windowtext;}
span.xxemailstyle23
{mso-style-name:x_xemailstyle23;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.xemailstyle28
{mso-style-name:x_emailstyle28;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.EmailStyle33
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:610817929;
mso-list-template-ids:-603314108;}
@list l0:level1
{mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level2
{mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level3
{mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level4
{mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level5
{mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level6
{mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level7
{mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level8
{mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l0:level9
{mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1
{mso-list-id:1423646984;
mso-list-template-ids:-933876240;}
@list l1:level1
{mso-level-tab-stop:.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level2
{mso-level-tab-stop:1.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level3
{mso-level-tab-stop:1.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level4
{mso-level-tab-stop:2.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level5
{mso-level-tab-stop:2.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level6
{mso-level-tab-stop:3.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level7
{mso-level-tab-stop:3.5in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level8
{mso-level-tab-stop:4.0in;
mso-level-number-position:left;
text-indent:-.25in;}
@list l1:level9
{mso-level-tab-stop:4.5in;
mso-level-number-position:left;
text-indent:-.25in;}
ol
{margin-bottom:0in;}
ul
{margin-bottom:0in;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple">
<div class="WordSection1">
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Attila and Peter,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I attempted to use the ‘so-reuseport’ option, but when I add it to my syslog-ng.conf file:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">source s_net { syslog(ip(0.0.0.0) transport("udp") so-reuseport(1) port(514)); };<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I end up getting this error:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Error parsing afsocket, inner-src plugin so-reuseport not found<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">so-reuseport(1)<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">^^^^^^^^^^^^<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">My version of syslog-ng is:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">syslog-ng-3.18.1-1.el7.x86_64<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Is this option on a later version? Or is this not available on the Open source version of syslog-ng.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Next, I attempted to use TCP instead of UDP. I put this into my syslog-ng.conf file:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">source s_net {<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> syslog(ip(0.0.0.0) transport("udp") port(514));<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> syslog(ip(0.0.0.0) transport("tcp") port(601)); };<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">I updated the rsyslog.conf file on my sending system to use TCP and port 601. But it does not work and I get these messages in my messages file on the syslog-ng
server:<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Syslog connection accepted; fd='35', client='AF_INET(10.146.70.155:52760)', local='AF_INET(0.0.0.0:601)'<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Invalid frame header; header=''<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Syslog connection closed; fd='35', client='AF_INET(10.146.70.155:52760)', local='AF_INET(0.0.0.0:601)'<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Bryan Klimek<o:p></o:p></span></p>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:blue"><o:p> </o:p></span></p>
</div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"><o:p> </o:p></span></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu]
<b>On Behalf Of </b>Attila Szakacs (aszakacs)<br>
<b>Sent:</b> Monday, July 1, 2019 6:18 AM<br>
<b>To:</b> 'Syslog-ng users' and developers' mailing list'<br>
<b>Subject:</b> [EXTERNAL] Re: [syslog-ng] Missing messages<o:p></o:p></span></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">Hi Bryan,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">Yes, you can use the same port for UDP and TCP.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">It would look something like this, in your config:<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Consolas;color:black">source s_net {</span><span style="font-family:"Calibri","sans-serif";color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Consolas;color:black"> syslog(ip(0.0.0.0) transport("udp") port(514));</span><span style="font-family:"Calibri","sans-serif";color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:Consolas;color:black"> <span style="background:white">syslog(ip(0.0.0.0) transport("tcp") port(514));</span></span><span style="font-family:"Calibri","sans-serif";color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt"><span style="font-family:Consolas;color:black">};</span><span style="font-family:"Calibri","sans-serif";color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black;background:white">Syslog-ng also has an option, which might be interesting for you as an alternative/addition to your architecture change: so-reuseport().</span><span style="font-family:"Calibri","sans-serif";color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black;background:white">It enables SO_REUSEPORT on systems that support it. </span><span style="font-family:"Calibri","sans-serif";color:black">When enabled, the kernel allows multiple
UDP sockets to be bound to the same port, and the kernel load-balances incoming UDP datagrams to the sockets.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black;background:white">You can read more about it here, under the so-reuseport() section: <a href="https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.22/administration-guide/25#TOPIC-1209161">https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.22/administration-guide/25#TOPIC-1209161</a></span><span style="font-family:"Calibri","sans-serif";color:black"><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">Best regards,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="font-family:"Calibri","sans-serif";color:black">Attila<o:p></o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="98%" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of
Klimek, Bryan J. <bklimek@mayo.edu><br>
<b>Sent:</b> Saturday, June 29, 2019 9:01 PM<br>
<b>To:</b> 'Syslog-ng users' and developers' mailing list'<br>
<b>Subject:</b> Re: [syslog-ng] Missing messages</span> <o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div style="border:solid #9C6500 1.0pt;padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="MsoNormal" style="line-height:12.0pt;background:#FFEB9C"><b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#9C6500">CAUTION:</span></b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:black"> This email
originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Attila,</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Thank you for that reminder that UDP is not reliable. As a quick test I added a line to the /etc/rsyslog.conf file the server that has the missing messages
and sent it to my test syslog-ng server that sees much less traffic. Now with the message traffic going to both my normal centralized syslog-ng server and my basically idle test syslog-ng server, I can see a difference. I seem to be dropping quite a few messages.
Not just the one message that was found to be missing.</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">What are the ramifications of changing my "s_net" source from UPD to TCP? Do I need to reconfigure the thousands of hosts that are sending to my centralized
syslog server to send using TCP. </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D">Can I configure both transports (UDP and TCP) on the same “s_net” source? If this were possible, this would seem to be an easy way to continue to support UDP
and transition over time to the TCP transport.</span><o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<p class="xxxmsonormal">Bryan Klimek<o:p></o:p></p>
<p class="xxxmsonormal">Mayo Clinic<o:p></o:p></p>
<p class="xmsonormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="xmsonormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu]
<b>On Behalf Of </b>Attila Szakacs (aszakacs)<br>
<b>Sent:</b> Thursday, June 27, 2019 3:25 AM<br>
<b>To:</b> 'syslog-ng@lists.balabit.hu'<br>
<b>Subject:</b> [EXTERNAL] Re: [syslog-ng] Missing messages</span><o:p></o:p></p>
</div>
</div>
<p class="xmsonormal"> <o:p></o:p></p>
<div>
<p class="xmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">Hi Bryan,</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-family:"Calibri","sans-serif";color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">If I understand correctly, the message, which got lost, was expected to come from the "s_net" source.</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">If that is the case, it is normal to lose logs there, as UDP does not provide reliable data transmission.</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-family:"Calibri","sans-serif";color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">Logging services have different solutions to this problem, syslog-ng Premium Edition has Advanced Log Transfer Protocol (ALTP).</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">You can read more about it here: <a href="https://support.oneidentity.com/technical-documents/syslog-ng-premium-edition/7.0.14/administration-guide/58#TOPIC-1187363">https://support.oneidentity.com/technical-documents/syslog-ng-premium-edition/7.0.14/administration-guide/58#TOPIC-1187363</a></span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-family:"Calibri","sans-serif";color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">If I misunderstood the problem, please correct me.</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-family:"Calibri","sans-serif";color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">Have a nice day!
</span><span style="font-family:"Segoe UI Symbol","sans-serif";color:black">🙂</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-family:"Calibri","sans-serif";color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">Best regards,</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">Attila</span><o:p></o:p></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="98%" align="center">
</div>
<div id="x_divRplyFwdMsg">
<p class="xmsonormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of
Klimek, Bryan J. <bklimek@mayo.edu><br>
<b>Sent:</b> Wednesday, June 26, 2019 3:50 PM<br>
<b>To:</b> 'syslog-ng@lists.balabit.hu'<br>
<b>Subject:</b> Re: [syslog-ng] Missing messages</span> <o:p></o:p></p>
<div>
<p class="xmsonormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div style="border:solid #9C6500 1.0pt;padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="xmsonormal" style="line-height:12.0pt;background:#FFEB9C"><b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#9C6500">CAUTION:</span></b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:black"> This email
originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.</span><o:p></o:p></p>
</div>
<p class="xmsonormal"> <o:p></o:p></p>
<div>
<div>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">Attila,</span><o:p></o:p></p>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">Thank you for your response. I enabled the "stats-level(1)" so I can see the statistics. But the “dropped” counters are all zero.
</span><o:p></o:p></p>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black"> </span><o:p></o:p></p>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">I’ve attached the syslog-ng.conf file for review.</span><o:p></o:p></p>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black"> </span><o:p></o:p></p>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">This syslog-ng instance receives syslog from 2000+ hosts. We create a separate file for every server for every day. We also fork the data to one consolidated file (all traffic
from all hosts into 1 file). We also fork a 3<sup>rd</sup> copy to our SEIM (Security Event Information Management) solution.</span><o:p></o:p></p>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black"> </span><o:p></o:p></p>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">The missing message does not appear in the individual host file, nor does it appear in the consolidate file of all hosts. I’ve never bothered to check with our SEIM people to
see if they are getting the missing message. I just assume they are not.</span><o:p></o:p></p>
<p class="xxmsonormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<div>
<p class="xxmsonormal"><b><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:navy">Bryan Klimek</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:blue">
<br>
</span><span style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:black">Phone: 507-284-9396</span><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:blue">
<br>
</span><span lang="PT-BR" style="font-size:10.0pt;font-family:"Verdana","sans-serif";color:black">E-mail:
<a href="mailto:klimek.bryan@mayo.edu">klimek.bryan@mayo.edu</a></span><span lang="PT-BR" style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:blue">
</span><o:p></o:p></p>
</div>
<p class="xxmsonormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D"> </span><o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in">
<p class="xxmsonormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu]
<b>On Behalf Of </b>Attila Szakacs (aszakacs)<br>
<b>Sent:</b> Monday, June 24, 2019 8:40 AM<br>
<b>To:</b> 'syslog-ng@lists.balabit.hu'<br>
<b>Subject:</b> [EXTERNAL] Re: [syslog-ng] Missing messages</span><o:p></o:p></p>
</div>
</div>
<p class="xxmsonormal"> <o:p></o:p></p>
<div>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">Hi Bryan,</span><o:p></o:p></p>
</div>
<div>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">Thank you for using the syslog-ng mailing list!
</span><span style="font-family:"Segoe UI Symbol","sans-serif";color:black">🙂</span><o:p></o:p></p>
</div>
<div>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">Message drop could happen for several reasons. I couple reasons, that suddenly comes up to me:</span><o:p></o:p></p>
</div>
<div>
<ol style="margin-top:0in" start="1" type="1">
<li class="MsoNormal" style="color:black;mso-list:l1 level1 lfo1"><span style="font-family:"Calibri","sans-serif"">The log path has a filter or parser, which does not match for the message.</span><o:p></o:p></li><li class="MsoNormal" style="color:black;mso-list:l1 level1 lfo1"><span style="font-family:"Calibri","sans-serif"">The log path has a source, which has a built-in parser (syslog for example), and the message does not match the protocol.</span><o:p></o:p></li><li class="MsoNormal" style="color:black;mso-list:l1 level1 lfo1"><span style="font-family:"Calibri","sans-serif"">Flow-control is not enabled and the destination is not alive for a longer period of time.</span><o:p></o:p></li><li class="MsoNormal" style="color:black;mso-list:l1 level1 lfo1"><span style="font-family:"Calibri","sans-serif"">Flow-control is configured incorrectly.</span><o:p></o:p></li><li class="MsoNormal" style="color:black;mso-list:l1 level1 lfo1"><span style="font-family:"Calibri","sans-serif"">...</span><o:p></o:p></li></ol>
<div>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">Saying so, it is hard to come up with one general way to investigate this.</span><o:p></o:p></p>
</div>
<div>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">However, I can give you some tips:</span><o:p></o:p></p>
</div>
<div>
<ol style="margin-top:0in" start="1" type="1">
<li class="MsoNormal" style="color:black;mso-list:l0 level1 lfo2"><span style="font-family:"Calibri","sans-serif"">You can set "stats-level(1)" in the global options and use "sbin/syslog-ng-ctl stats", then look for the "dropped" counters.</span><o:p></o:p></li><li class="MsoNormal" style="color:black;mso-list:l0 level1 lfo2"><span style="font-family:"Calibri","sans-serif"">You can start syslog-ng in debug mode (./sbin/syslog-ng -Fedtv) and look for the following logs: "Destination queue full, dropping message;" or
"UNMATCHED".</span><o:p></o:p></li><li class="MsoNormal" style="color:black;mso-list:l0 level1 lfo2"><span style="font-family:"Calibri","sans-serif"">Check if flow-control is enabled and configured properly.</span><o:p></o:p></li><li class="MsoNormal" style="color:black;mso-list:l0 level1 lfo2"><span style="font-family:"Calibri","sans-serif"">If you do not want to use flow-control, you can use disk-queue alternatively.</span><o:p></o:p></li></ol>
<div>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">If you could share some parts of your setup and config, where the problem happens, we could give you more insight.</span><o:p></o:p></p>
</div>
</div>
<div>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">Best regards,</span><o:p></o:p></p>
</div>
<div>
<p class="xxmsonormal"><span style="font-family:"Calibri","sans-serif";color:black">Attila</span><o:p></o:p></p>
</div>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="3" width="98%" align="center">
</div>
<div id="x_x_divRplyFwdMsg">
<p class="xxmsonormal"><b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black">From:</span></b><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:black"> syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of
Klimek, Bryan J. <bklimek@mayo.edu><br>
<b>Sent:</b> Monday, June 24, 2019 3:05 PM<br>
<b>To:</b> 'syslog-ng@lists.balabit.hu'<br>
<b>Subject:</b> [syslog-ng] Missing messages</span> <o:p></o:p></p>
<div>
<p class="xxmsonormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div style="border:solid #9C6500 1.0pt;padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="xxmsonormal" style="line-height:12.0pt;background:#FFEB9C"><b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:#9C6500">CAUTION:</span></b><span style="font-size:10.0pt;font-family:"Calibri","sans-serif";color:black"> This email
originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.</span><o:p></o:p></p>
</div>
<p class="xxmsonormal"> <o:p></o:p></p>
<div>
<div>
<p class="xxxmsonormal">First time poster, so be gentle.<o:p></o:p></p>
<p class="xxxmsonormal"> <o:p></o:p></p>
<p class="xxxmsonormal">We run syslog-ng in our environment as a centralized syslog manager for the archiving of syslog data. We have over 2000+ Unix/Linux systems sending their syslog data with a daily ingest rate of about 10GB per day.<o:p></o:p></p>
<p class="xxxmsonormal"> <o:p></o:p></p>
<p class="xxxmsonormal">It was recently pointed out to me that one particular message from one specific host is not getting persisted to the files on our syslog-ng server all the time. That is to say, it is intermittent.<o:p></o:p></p>
<p class="xxxmsonormal"> <o:p></o:p></p>
<p class="xxxmsonormal">If one person can find one message that is not making into our syslog-ng archive, I can only assume that we are dropping other messages as well.<o:p></o:p></p>
<p class="xxxmsonormal"> <o:p></o:p></p>
<p class="xxxmsonormal">How can I debug if and when messages are being lost and not making into the files on my centralized syslog server?<o:p></o:p></p>
<p class="xxxmsonormal"> <o:p></o:p></p>
<p class="xxxmsonormal">Bryan Klimek<o:p></o:p></p>
<p class="xxxmsonormal">Mayo Clinic<o:p></o:p></p>
<p class="xxxmsonormal"> <o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>