<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<meta name="Generator" content="Microsoft Word 15 (filtered medium)">
<!--[if !mso]><style>v\:* {behavior:url(#default#VML);}
o\:* {behavior:url(#default#VML);}
w\:* {behavior:url(#default#VML);}
.shape {behavior:url(#default#VML);}
</style><![endif]--><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Consolas;
panose-1:2 11 6 9 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
p.xmsonormal, li.xmsonormal, div.xmsonormal
{mso-style-name:x_msonormal;
margin:0cm;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
span.EmailStyle23
{mso-style-type:personal-reply;
font-family:"Calibri",sans-serif;
color:windowtext;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:72.0pt 72.0pt 72.0pt 72.0pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]-->
</head>
<body lang="EN-US" link="blue" vlink="purple" style="word-wrap:break-word">
<div class="WordSection1">
<p class="MsoNormal"><span style="color:#7030A0">Hi Gabor,<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#7030A0"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#7030A0">Thank you for you feedback.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#7030A0"><o:p> </o:p></span></p>
<p class="MsoNormal"><i><span style="color:black">Can you share the config, when the issue cannot be seen?<o:p></o:p></span></i></p>
<p class="MsoNormal"><span style="color:#7030A0">I am sending the configuration in attachment.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#7030A0"><o:p> </o:p></span></p>
<p class="MsoNormal"><i><span style="color:black">I would still like to see 2 "syslog-ng-ctl stats" outputs when the issue happens.<o:p></o:p></span></i></p>
<p class="MsoNormal"><span style="color:#7030A0">The issue is hard to reproduce, the next time the error is seen, I try to run it.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#7030A0">But how can I run syslog-ng-ctl stats for the 2<sup>nd</sup> syslog-ng instance?<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:Consolas;color:#7030A0">root@machine:/~# ps -ewfH | grep syslog-ng<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:Consolas;color:#7030A0">root 2582 1 0 09:06 ? 00:00:34 /usr/sbin/syslog-ng -F --caps cap_net_bind_service,cap_net_broadcast,cap_net_raw,cap_dac_read_search,cap_chown,cap_fowner=p
cap_dac_override,cap_syslog=ep<o:p></o:p></span></p>
<p class="MsoNormal"><span style="font-size:8.0pt;font-family:Consolas;color:#7030A0">root 4018 1 0 09:07 ? 00:00:00 /usr/sbin/syslog-ng -F --cfgfile=/etc/syslog-ng/mgmt-syslog-ng.conf --pidfile=/var/lib/syslog-ng/mgmt-syslog-ng.pid --persist-file=/var/lib/syslog-ng/mgmt-syslog-ng.persist
--control=/var/lib/syslog-ng/mgmt-syslog-ng.ctl<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#7030A0">syslog-ng-ctl, seems to only show stats for job 2582.<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#7030A0"><o:p> </o:p></span></p>
<p class="MsoNormal"><span style="color:#7030A0">Regards, Alex<o:p></o:p></span></p>
<p class="MsoNormal"><span style="color:#7030A0"><o:p> </o:p></span></p>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="MsoNormal"><b>From:</b> Gabor Nagy (gnagy) <Gabor.Nagy@oneidentity.com>
<br>
<b>Sent:</b> 29 de março de 2022 12:48<br>
<b>To:</b> Alexandre Santos <ASantos@infinera.com>; Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu><br>
<b>Subject:</b> Re: Local sources seem not to be working<o:p></o:p></p>
</div>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<p class="MsoNormal"><span style="color:black">Hi Alex,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Using regular disk-buffer vs. using reliable disk-buffer shouldn't cause symptoms like that. It sounds like reliable(yes) would turn on a flow-control-like behaviour, which it doesn't.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">(And as you said it only affects local sources).<br>
The main difference between the two kinds of disk-buffers is, that while reliable disk-buffer write every message to the disk-buffer, a normal disk-buffer has memory-only buffers for performance reasons (and flow-control reasons too).<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">You can still lose logs with a reliable disk-buffer if no flow-control is used: when the disk-buffer has reached it's maximum size and new messages keep arriving, then syslog-ng drops those messages.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">We have more detailed documentation about disk-buffers in the admin guide, where you can see the structure of disk-buffers:<br>
<a href="https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.syslog-ng.com%2Ftechnical-documents%2Fdoc%2Fsyslog-ng-open-source-edition%2F3.36%2Fadministration-guide%2F61%23TOPIC-1768724&data=04%7C01%7CASantos%40infinera.com%7C888d04fa66b245d8a46008da117a0a88%7C285643de5f5b4b03a1530ae2dc8aaf77%7C1%7C1%7C637841513109614956%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=7M%2BxYTvsi1hu01apm%2BE1gNndB%2BZVzPnjbfvkWiOOBdA%3D&reserved=0">https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.36/administration-guide/61#TOPIC-1768724</a><o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><o:p> </o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Can you share the config, when the issue cannot be seen?<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">I would still like to see 2 "syslog-ng-ctl stats" outputs when the issue happens.<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black"><br>
Regards,<o:p></o:p></span></p>
</div>
<div>
<p class="MsoNormal"><span style="color:black">Gabor<o:p></o:p></span></p>
</div>
<div class="MsoNormal" align="center" style="text-align:center">
<hr size="2" width="98%" align="center">
</div>
<div id="divRplyFwdMsg">
<p class="MsoNormal"><b><span style="color:black">From:</span></b><span style="color:black"> Alexandre Santos <<a href="mailto:ASantos@infinera.com">ASantos@infinera.com</a>><br>
<b>Sent:</b> Monday, March 28, 2022 13:45<br>
<b>To:</b> Gabor Nagy (gnagy) <<a href="mailto:Gabor.Nagy@oneidentity.com">Gabor.Nagy@oneidentity.com</a>>; Syslog-ng users' and developers' mailing list <<a href="mailto:syslog-ng@lists.balabit.hu">syslog-ng@lists.balabit.hu</a>><br>
<b>Subject:</b> RE: Local sources seem not to be working</span> <o:p></o:p></p>
<div>
<p class="MsoNormal"> <o:p></o:p></p>
</div>
</div>
<div>
<div style="border:solid #9C6500 1.0pt;padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="MsoNormal" style="line-height:12.0pt;background:#FFEB9C"><b><span style="font-size:10.0pt;color:#9C6500">CAUTION:</span></b><span style="font-size:10.0pt;color:black"> This email originated from outside of the organization. Do not follow guidance,
click links, or open attachments unless you recognize the sender and know the content is safe.<o:p></o:p></span></p>
</div>
<p class="MsoNormal"><o:p> </o:p></p>
<div>
<div>
<p class="xmsonormal"><span style="color:#7030A0">Hi Gabor,</span><o:p></o:p></p>
<p class="xmsonormal"><span style="color:#7030A0"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="color:#7030A0">“<i>This is strange: the d_localfile destinations (<span style="background:white">as well as the vrf-socket destination "d_mgmt_vrf_socket"</span>) receive messages from the syslog() source, but not from the
internal() or system() sources?”</i></span><o:p></o:p></p>
<p class="xmsonormal"><span style="color:#7030A0">Yes.</span><o:p></o:p></p>
<p class="xmsonormal"><span style="color:#7030A0"> </span><o:p></o:p></p>
<p class="xmsonormal"><i><span style="color:#7030A0">And the issue vanishes when "d_mgmt_vrf_socket" destination is removed?</span></i><o:p></o:p></p>
<p class="xmsonormal"><span style="color:#7030A0">Yes.</span><o:p></o:p></p>
<p class="xmsonormal"><span style="color:#7030A0"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="color:#7030A0">I could not test the 2 last suggestions that you made.</span><o:p></o:p></p>
<p class="xmsonormal"><span style="color:#7030A0"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="color:#7030A0">We did however another test, which was to
<b>remove the reliable</b> option from d_mgmt_vrf_socket, and it seems the <b>problem is not seen again</b>.</span><o:p></o:p></p>
<p class="xmsonormal"><span style="color:#7030A0"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="color:#7030A0">Besides from what it is written in the manual, in other which cases/conditions can syslog-ng loose logs?</span><o:p></o:p></p>
<p class="xmsonormal"><span style="color:#7030A0"> </span><o:p></o:p></p>
<table class="MsoNormalTable" border="0" cellspacing="0" cellpadding="0" width="100%" style="width:100.0%">
<tbody>
<tr>
<td colspan="2" style="padding:.75pt .75pt .75pt .75pt">
<p class="xmsonormal"><span style="color:#7030A0">reliable() </span><o:p></o:p></p>
</td>
</tr>
<tr>
<td style="padding:.75pt .75pt .75pt .75pt">
<p class="xmsonormal"><span style="color:#7030A0">Type:</span><o:p></o:p></p>
</td>
<td style="padding:.75pt .75pt .75pt .75pt">
<p class="xmsonormal"><span style="color:#7030A0">yes|no</span><o:p></o:p></p>
</td>
</tr>
<tr>
<td style="padding:.75pt .75pt .75pt .75pt">
<p class="xmsonormal"><span style="color:#7030A0">Default:</span><o:p></o:p></p>
</td>
<td style="padding:.75pt .75pt .75pt .75pt">
<p class="xmsonormal"><span style="color:#7030A0">no</span><o:p></o:p></p>
</td>
</tr>
<tr>
<td colspan="2" style="padding:.75pt .75pt .75pt .75pt">
<p class="xmsonormal"><span style="color:#7030A0">Description: If set to yes, syslog-ng OSE cannot lose logs in case of reload/restart, unreachable destination or syslog-ng OSE crash. This solution provides a slower, but reliable disk-buffer option. It is created
and initialized at startup and gradually grows as new messages arrive. If set to no, the normal disk-buffer will be used. This provides a faster, but less reliable disk-buffer option.</span><o:p></o:p></p>
</td>
</tr>
</tbody>
</table>
<p class="xmsonormal"><span style="color:#7030A0"> </span><o:p></o:p></p>
<p class="xmsonormal"><span style="color:#7030A0">Thanks in advance,</span><o:p></o:p></p>
<p class="xmsonormal"><span style="color:#7030A0">Alex</span><o:p></o:p></p>
<p class="xmsonormal"> <o:p></o:p></p>
<div>
<div style="border:none;border-top:solid #E1E1E1 1.0pt;padding:3.0pt 0cm 0cm 0cm">
<p class="xmsonormal"><b>From:</b> Gabor Nagy (gnagy) <<a href="mailto:Gabor.Nagy@oneidentity.com">Gabor.Nagy@oneidentity.com</a>>
<br>
<b>Sent:</b> 25 de março de 2022 14:44<br>
<b>To:</b> Alexandre Santos <<a href="mailto:ASantos@infinera.com">ASantos@infinera.com</a>>; Syslog-ng users' and developers' mailing list <<a href="mailto:syslog-ng@lists.balabit.hu">syslog-ng@lists.balabit.hu</a>><br>
<b>Subject:</b> Re: Local sources seem not to be working<o:p></o:p></p>
</div>
</div>
<p class="xmsonormal"> <o:p></o:p></p>
<div style="border:solid #9C6500 1.0pt;padding:2.0pt 2.0pt 2.0pt 2.0pt">
<p class="xmsonormal" style="line-height:12.0pt;background:#FFEB9C"><b><span style="font-size:10.0pt;color:#9C6500">CAUTION:</span></b><span style="font-size:10.0pt;color:black"> This email originated from outside of the organization. Do not click links or
open attachments unless you recognize the sender and know the content is safe.</span><o:p></o:p></p>
</div>
<p class="xmsonormal"> <o:p></o:p></p>
<div>
<div>
<p class="xmsonormal"><span style="color:black">Hi Alex,</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="color:black"><br>
Sorry I haven't answered yet. I'll have a few ideas I would like to try out next week.</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="color:black"><br>
This is strange: the d_localfile destinations (<span style="background:white">as well as the vrf-socket destination "d_mgmt_vrf_socket"</span>) receive messages from the syslog() source, but not from the internal() or system() sources?</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="color:black">And the issue vanishes when "d_mgmt_vrf_socket" destination is removed?</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="color:black">If it would be soft flow-control, then the syslog() source would be suspended too.</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="color:black;background:white">Just a tip: would you switch out the unix-dgram() destination to syslog() destination, please? Maybe that's not possible with the VRF in-place...</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="color:black"><br>
In the stats output, do you see an increased number of dropped messages?</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="color:black"> </span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="color:black">I would still suggest increasing the 4MB disk-buffer. You should make an estimation of how long could the mgmt syslog-ng be down (i.e not receiving from the unix-dgram), what is the average incoming EPS and an
average message size, that could give a hint about the required disk-buffer size.</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal"><span style="color:black"><br>
Regards,</span><o:p></o:p></p>
</div>
<div>
<p class="xmsonormal" style="margin-bottom:12.0pt"><span style="color:black">Gabor</span><o:p></o:p></p>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>