Hi Justin.<br><br>Thank you for your help. In the project I'm in, I won't have the time to do it in a low level (kernel) as you refer and as I also thought it would be great if Linux already had a device like this (a rotation file with parameters of lines and/or size).
<br><br>So, my solution will be something like a buffer in a daemon and for each new line, I can do a shift on the buffer. I will implement a behaviour more or less like this. <br><br>I like your suggestion of the linked lists, but a buffer will also do the way since I never want to change the order.. I mean, no need to change the pointers NEXT and and PREVIOUS. A simple shift of elements will make sense..
<br><br>Many regards for your so complete solution.<br>Bruno.<br><br><div><span class="gmail_quote">On 3/27/07, <b class="gmail_sendername">Justin Randall</b> <<a href="mailto:jrandall@comwave.net">jrandall@comwave.net
</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div link="blue" vlink="blue" lang="EN-US">
<div>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;">In fact, you
could even write this as a kernel module (which would give you direct access to
the list.h kernel header) and implement this behaviour right under the hood of
the OS rather than in Syslog-NG.</span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;"> </span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;">Basically, using
the Linux VFS and kernel module subsystems, you can create device files in
which the "write" operations behave in the manner listed below (writing
to a circular list implementation of X number of nodes). This would then
allow Syslog-NG to continuously write to the device file, and let the device
file's implementation operate in the ring buffer manner outlined below.</span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;"> </span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;">I hope this
helps,</span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;"> </span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;">Justin.</span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;"> </span></font></p>
<div>
<div style="text-align: center;" align="center"><font face="Times New Roman" size="3"><span style="font-size: 12pt;">
<hr align="center" size="2" width="100%">
</span></font></div>
<p><b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma; font-weight: bold;">From:</span></font></b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"> Justin Randall
<br>
<b><span style="font-weight: bold;">Sent:</span></b> Tuesday, March 27, 2007 5:47
PM<span class="q"><br>
<b><span style="font-weight: bold;">To:</span></b> 'Syslog-ng
users' and developers' mailing list'<br></span>
<b><span style="font-weight: bold;">Subject:</span></b> RE: [syslog-ng] Log
rotate without log rotate.</span></font></p>
</div><div><span class="e" id="q_111959583802b733_3">
<p><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;">Depending on the
language you are using, if you have access to pointers (if you are using C/C++)
you can very efficiently implement this behaviour using a circular ring buffer.
Essentially this would involve a double-linked list implementation in which
the head node's "prev" member references the tail node of the
list.</span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;"> </span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;">The list
implementation used in the Linux kernel (list.h in the kernel header files)
uses this type of list. Using this list implementation, you can have your
circular buffer by doing the following:</span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;"> </span></font></p>
<p style="margin-left: 20.25pt; text-indent: -0.25in;"><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;"><span>-<font face="Times New Roman" size="1"><span>
</span></font></span></span></font><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;">Create a list with 100 nodes, each node representing a
"line" of logging text.</span></font></p>
<p style="margin-left: 20.25pt; text-indent: -0.25in;"><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;"><span>-<font face="Times New Roman" size="1"><span>
</span></font></span></span></font><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;">Create an integrator-type pointer used to traverse the list.</span></font></p>
<p style="margin-left: 20.25pt; text-indent: -0.25in;"><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;"><span>-<font face="Times New Roman" size="1"><span>
</span></font></span></span></font><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;">Each node contains a "next" and a "prev"
pointer which reference the next or previous node in the list.</span></font></p>
<p style="margin-left: 20.25pt; text-indent: -0.25in;"><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;"><span>-<font face="Times New Roman" size="1"><span>
</span></font></span></span></font><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;">When writing a line, modify the text line for the node currently
referenced by the integrator pointer, and the advance the iterator to the value
of the "next" pointer for the current node.</span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;"> </span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;">Using this
logic, the middle-application will be very compact, as it does not need any
program logic to determine if it is in need to dump old data for new data.</span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;"> </span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;">You may have
already gone down this path, and if so I apologize for the long-winded email,
though I hope this is helpful in writing a light-weight middle-app.</span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;"> </span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;">Regards,</span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;"> </span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;">Justin.</span></font></p>
<p><font color="blue" face="Book Antiqua" size="2"><span style="font-size: 10pt; color: blue;"> </span></font></p>
<div>
<div style="text-align: center;" align="center"><font face="Times New Roman" size="3"><span style="font-size: 12pt;">
<hr align="center" size="2" width="100%">
</span></font></div>
<p><b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma; font-weight: bold;">From:</span></font></b><font face="Tahoma" size="2"><span style="font-size: 10pt; font-family: Tahoma;"> Bruno Vieira
[mailto:<a href="mailto:giuliapo@gmail.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">giuliapo@gmail.com</a>] <br>
<b><span style="font-weight: bold;">Sent:</span></b> Saturday, March 17, 2007
12:58 PM<br>
<b><span style="font-weight: bold;">To:</span></b> Syslog-ng
users' and developers' mailing list<br>
<b><span style="font-weight: bold;">Subject:</span></b> Re: [syslog-ng] Log
rotate without log rotate.</span></font></p>
</div>
<p><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font></p>
<p style="margin-bottom: 12pt;"><font face="Times New Roman" size="3"><span style="font-size: 12pt;">Hi there. Thank you for
your opinion.<br>
<br>
I have found that this really is a feature well needed by many embeddedsystems.<br>
<br>
I will implement or reuse a component that makes this kind of rotation... the
conclusion i got was that what I really needed was a file that would implement
a buffer... so, the only way is making a daemon that does this for me.. and it
may or may not save it's buffer on a destination file. <br>
<br>
It would be great if Linux had a driver that would do this on it's own. Or at
least a driver on the project openwrt (linux for embedded systems).<br>
<br>
Well, I have the solution now :)<br>
<br>
Thanks you all for your contribution. <br>
Many regards,<br>
Bruno.<br>
<br>
<br>
<br>
</span></font></p>
<div>
<p><span><font face="Times New Roman" size="3"><span style="font-size: 12pt;">On 17 Mar 2007 16:25:32 +0000, <b><span style="font-weight: bold;">Bryan Henderson</span></b> <<a href="mailto:bryanh@giraffe-data.com" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">
bryanh@giraffe-data.com </a>> wrote:</span></font></span></p>
<p style="margin-bottom: 12pt;"><font face="Times New Roman" size="3"><span style="font-size: 12pt;">>I have a solution.. but
I think there can be a better one. My solution is: <br>
>having a midlle program between syslog-ng and the destination file. And for<br>
>every message received, the program would verify if the destination file<br>
>(p.e. auth.log) already has 100 lines. If so, deleted the last line, make <br>
>all others one ENTER down and that append the new line to the beginning of<br>
>the file.<br>
<br>
It would probably be better to maintain the file in circular fashion:<br>
allocate 8K in the beginning, then write messages from beginning to <br>
end, then wrap around and write from the beginning again, always<br>
keeping track (in a header of the file) of where you last wrote. Use<br>
a special program that understands this file format to translate it to<br>
a regular stream that you can use normally. <br>
<br>
Lots of systems have programs that do this. I looked on Freshmeat
for<br>
a publicly available one and found 'cupyvei', though I don't much care<br>
for the details of this implementation. It would be easy to write a <br>
program of your own.<br>
<br>
I also think a built in log destination of this type<br>
(e.g. "file_circular") would be an excellent addition to
syslog-ng. I<br>
think this type of logging is a common requirement. <br>
<br>
--<br>
Bryan
Henderson
San Jose, California<br>
_______________________________________________<br>
syslog-ng maillist - <a href="mailto:syslog-ng@lists.balabit.hu" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">syslog-ng@lists.balabit.hu </a><br>
<a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Frequently asked questions at <a href="http://www.campin.net/syslog-ng/faq.html" target="_blank" onclick="return top.js.OpenExtLink(window,event,this)">http://www.campin.net/syslog-ng/faq.html
</a></span></font></p>
</div>
<p><font face="Times New Roman" size="3"><span style="font-size: 12pt;"> </span></font></p>
</span></div></div>
</div>
<br>_______________________________________________<br>syslog-ng maillist - <a onclick="return top.js.OpenExtLink(window,event,this)" href="mailto:syslog-ng@lists.balabit.hu">syslog-ng@lists.balabit.hu</a><br><a onclick="return top.js.OpenExtLink(window,event,this)" href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">
https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>Frequently asked questions at <a onclick="return top.js.OpenExtLink(window,event,this)" href="http://www.campin.net/syslog-ng/faq.html" target="_blank">http://www.campin.net/syslog-ng/faq.html
</a><br><br><br></blockquote></div><br>