<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
Hi Charles,</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
<span style="margin: 0px">In syslog-ng you can have file, pipe, etc... destinations with templated path. This template can contain time-related values, for example the hour, minute, second when the message was sent.</span><br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
We can imagine a scenario, where we want to store the incoming messages in separate files for every minute, so we set the template accordingly. Days gone by, and we opened so many file descriptors, that we reach the kernel FD limit.</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
To avoid situations like this, we introduced<span style="margin: 0px"> a global option called time-reap(). It will close the file descriptors, which were not used in a long time. You can set this timeout in seconds, and it is 60 by default.</span></div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
Unfortunately time-reap() has its flaws.</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
The first problem: It can only be set globally, for all destinations.</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
The second problem: It can not be disabled. We should be able to disable it, if we are sure, that no problematic templating is set for that destination.</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
The third problem: Instead of starting/resetting the timeout when we sent out a message to that destination, the timer is started when the FD is opened/reopened. Additionally, when the timeout runs out, we check if there is any pending message to that destination,
which might not be true, if we are not sending anything to that FD in that exact moment.</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
I plan to fix the first two problems, as they are relatively easy to fix. I will consult with the team.</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
Regarding your case, you can set time-reap to a really high value, but it still will not make sure, that your pipe will not be closed.</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
To find a workaround, could you tell me how impactful this problem is for you? Does sagan work well after the FD is reopened?</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
Best regards,</div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif">
Attila</div>
<br>
</div>
<div id="appendonsend"></div>
<div style="font-family:Calibri,Arial,Helvetica,sans-serif; font-size:12pt; color:rgb(0,0,0)">
<br>
</div>
<hr tabindex="-1" style="display:inline-block; width:98%">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" color="#000000" style="font-size:11pt"><b>From:</b> syslog-ng <syslog-ng-bounces@lists.balabit.hu> on behalf of Charles Eidsness <charles@ccxtechnologies.com><br>
<b>Sent:</b> Friday, June 21, 2019 6:22 PM<br>
<b>To:</b> syslog-ng@lists.balabit.hu<br>
<b>Subject:</b> [syslog-ng] Periodic "Destination timed out, reaping" and disconnection on a write fifo (pipe)</font>
<div> </div>
</div>
<div>
<div style="background-color:#FFEB9C; width:100%; border-style:solid; border-color:#9C6500; border-width:1pt; padding:2pt; font-size:10pt; line-height:12pt; font-family:'Calibri'; color:Black; text-align:left">
<span style="color:#9C6500; font-weight:bold">CAUTION:</span> This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.</div>
<br>
<div>
<div dir="ltr">
<div class="x_gmail_default" style="font-family:verdana,sans-serif">
<div dir="ltr">
<div class="x_gmail_default" style="font-family:verdana,sans-serif">I kind of feel like this is going to be a dumb question but I just can't figure this out.</div>
<div class="x_gmail_default" style="font-family:verdana,sans-serif"><br>
</div>
<div dir="ltr">I'm trying to get syslog-ng working with sagan, but every 60 seconds syslog-ng closes the fifo<span class="x_gmail_default" style="font-family:verdana,sans-serif"> it's writing to,</span> which closes the sagan connection to the fifo<span class="x_gmail_default" style="font-family:verdana,sans-serif">.
When running in verbose mode syslog-ng is </span>thro<span class="x_gmail_default" style="font-family:verdana,sans-serif">wing
</span>up a "<span style="font-family:verdana,sans-serif">Destination timed out, reaping<span class="x_gmail_default" style="font-family:verdana,sans-serif">..." message.</span></span></div>
<div dir="ltr"><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif"><br>
</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif">I simplified my syslog-ng configuration to:</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif"><br>
</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif">source s_sys {</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif"> system();<br>
};<br>
<br>
destination d_fifo {<br>
pipe(</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif"> "/tmp/test.fifo"</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif"> );<br>
};<br>
</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif"><br>
</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif">log {<br>
source(s_sys);<br>
destination(d_fifo);<br>
};</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif"><br>
</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif">And then run use cat to read the fifo:</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif">cat /tmp/test.fifo
<br>
</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif"><br>
</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif">It sends all of the log traffic every but every 60 seconds cat closes. Is this expected behavior? Am I missing something?</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif"><br>
</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif"><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif">I'm using syslog-ng version
3.19.1.</span></span></span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif"><br>
</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif">Thanks!</span></span></div>
<div><span style="font-family:verdana,sans-serif"><span class="x_gmail_default" style="font-family:verdana,sans-serif">Charles
<div class="x_gmail-adL"><br>
</div>
</span></span></div>
</div>
</div>
<div>
<div dir="ltr" class="x_gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div><span style="font-family:verdana,sans-serif">==<br>
</span></div>
<div><span style="font-family:verdana,sans-serif">Charles Eidsness<br>
</span></div>
<span style="font-family:verdana,sans-serif">Chief Engineer<br>
</span></div>
<span style="font-family:verdana,sans-serif"><b>CCX Technologies</b><br>
(613)791-9715</span></div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>