<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns="http://www.w3.org/TR/REC-html40">
<head>
<meta http-equiv=Content-Type content="text/html; charset=us-ascii">
<meta name=Generator content="Microsoft Word 11 (filtered medium)">
<style>
<!--
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{color:purple;
text-decoration:underline;}
p
{mso-margin-top-alt:auto;
margin-right:0cm;
mso-margin-bottom-alt:auto;
margin-left:0cm;
font-size:12.0pt;
font-family:"Times New Roman";}
span.EmailStyle17
{mso-style-type:personal-compose;
font-family:"Times New Roman";
color:windowtext;
font-weight:normal;
font-style:normal;
text-decoration:none none;}
@page Section1
{size:595.3pt 841.9pt;
margin:3.0cm 2.0cm 3.0cm 2.0cm;}
div.Section1
{page:Section1;}
-->
</style>
</head>
<body lang=DA link=blue vlink=purple>
<div class=Section1>
<p class=MsoNormal><font size=2 face="Times New Roman"><span style='font-size:
10.0pt'>Hello<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span style='font-size:
10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'>I am currently exploring the possibilitys with
flowcontrol from syslog-ng 2.0 on a freeBSD system. <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'>I have logs coming in from different sources to a
centralized logserver, where I want to save the logs in an archive, and also
read the logs into a database. What I seek to accomplish is that in case of
database breakdown, logs should still be written to the archive, and when the
database is up again, the missing logs should then automatically be written to
the database, all in chronological order.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'>The way I plan to do it, with the new possibilities
flow-control offer, is that incoming logs are piped to a script that do 2
things. One is to keep count of the linenumbers in the archivefiles, and add to
each log its own linenumber, before writting the log to its archivefile. The
second task is to pipe a copy of the log (with the line number added) back to
syslog-ng. Syslog-ng then pipes the log to another script that reads the
linenumber from the log, and compare it to the linenumber from the previous log
this script read. If the linenumbers are not consecutive, it means that some
logs where lost on the way from the archivescript to this accounterscript. It
therefore fetches the missing logs from the archive, identified by the
linenumbers, before reading from the pipe again. <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'>If I use flow-control on the logs coming in from the
sources and piped to the archive script, and no flow-control on the logs that
are piped back to syslog-ng and further on to the accounterscript, I accomplish
that logs are always written to the archive, even when the database fails, and
all logs are later written to database when it comes up again, since there is
account for all the logs through their linenumber. The basic idea being
that everything is piped under normal conditions, and I only read from the
archive in case of databaseproblems.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'>The flow-control works fine for the first part of the
process. I can have the archivescript write to the archive with no loss of
logs. I can also let the archive script write the same log both to the archive
and a pipe, when I place a script at the other end of the pipe to read the pipe,
and still not loose logs.. But when I try to write to a pipe that syslog-ng
shall read, the flowcontrol doesn’t seem to work. Log are lost on the way
to the archive script, even though its flowcontrolled. <o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'>My question is, if there is something I have
overlooked that might cause this behaviour. It works when I read the pipe from
a script of my own, but not when syslog-ng reads it. The reason I want
syslog-ng to read the pipe, and not simply a script of my own, is that
syslog-ng will read from the pipe, even if the database is down, where my
own script would block on the pipe, and consequently block for incoming logs,
since that part ( the other end of the pipe where the archivescript is) is
flow-controlled.<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'>I read somewhere that syslog-ng opens a pipe for both
reading and writing, but I don’t quite understand how this comply with
the nature of pipes. Can anyone elaborate on this? And could that be a problem in
my setup?<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=2 face="Times New Roman"><span style='font-size:10.0pt'>Kind Regards</span></font> <font
size=2><span style='font-size:10.0pt'><o:p></o:p></span></font></p>
<p class=MsoNormal><b><font size=2 face="Times New Roman"><span
style='font-size:10.0pt;font-weight:bold'>Steen Thomsen</span></font></b><font
size=1><span style='font-size:9.0pt'><o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span style='font-size:
10.0pt'>softwaredeveloper<o:p></o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span style='font-size:
10.0pt'><o:p> </o:p></span></font></p>
<p align=center style='text-align:center'><font size=3 face="Times New Roman"><span
style='font-size:12.0pt'> </span></font><font size=1><span
style='font-size:9.0pt'><o:p></o:p></span></font></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=3 face="Times New Roman"><span lang=EN-GB style='font-size:12.0pt'><o:p> </o:p></span></font></p>
<p class=MsoNormal><font size=2 face="Times New Roman"><span lang=EN-GB
style='font-size:10.0pt'><br>
<o:p></o:p></span></font></p>
<p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><font
size=3 face="Times New Roman"><span style='font-size:12.0pt'><a
href="http://www.bridicum.com/"
title="http://www.bridicum.com/ blocked::http://www.bridicum.com http://www.bridicum.com"></a></span></font><font
size=1><span style='font-size:9.0pt'><o:p></o:p></span></font></p>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><a href="http://www.bridicum.com/"
title="http://www.bridicum.com/ blocked::http://www.bridicum.com http://www.bridicum.com"></a></span><o:p></o:p></font></p>
<p class=MsoNormal><font size=3 face="Times New Roman"><span style='font-size:
12.0pt'><o:p> </o:p></span></font></p>
</div>
</body>
</html>