In fact, you could even write this as a kernel module (which would give you direct access to the list.h kernel header) and implement this behaviour right under the hood of the OS rather than in Syslog-NG. Basically, using the Linux VFS and kernel module subsystems, you can create device files in which the "write" operations behave in the manner listed below (writing to a circular list implementation of X number of nodes). This would then allow Syslog-NG to continuously write to the device file, and let the device file's implementation operate in the ring buffer manner outlined below. I hope this helps, Justin. _____ From: Justin Randall Sent: Tuesday, March 27, 2007 5:47 PM To: 'Syslog-ng users' and developers' mailing list' Subject: RE: [syslog-ng] Log rotate without log rotate. Depending on the language you are using, if you have access to pointers (if you are using C/C++) you can very efficiently implement this behaviour using a circular ring buffer. Essentially this would involve a double-linked list implementation in which the head node's "prev" member references the tail node of the list. The list implementation used in the Linux kernel (list.h in the kernel header files) uses this type of list. Using this list implementation, you can have your circular buffer by doing the following: - Create a list with 100 nodes, each node representing a "line" of logging text. - Create an integrator-type pointer used to traverse the list. - Each node contains a "next" and a "prev" pointer which reference the next or previous node in the list. - When writing a line, modify the text line for the node currently referenced by the integrator pointer, and the advance the iterator to the value of the "next" pointer for the current node. Using this logic, the middle-application will be very compact, as it does not need any program logic to determine if it is in need to dump old data for new data. You may have already gone down this path, and if so I apologize for the long-winded email, though I hope this is helpful in writing a light-weight middle-app. Regards, Justin. _____ From: Bruno Vieira [mailto:giuliapo@gmail.com] Sent: Saturday, March 17, 2007 12:58 PM To: Syslog-ng users' and developers' mailing list Subject: Re: [syslog-ng] Log rotate without log rotate. Hi there. Thank you for your opinion. I have found that this really is a feature well needed by many embeddedsystems. I will implement or reuse a component that makes this kind of rotation... the conclusion i got was that what I really needed was a file that would implement a buffer... so, the only way is making a daemon that does this for me.. and it may or may not save it's buffer on a destination file. It would be great if Linux had a driver that would do this on it's own. Or at least a driver on the project openwrt (linux for embedded systems). Well, I have the solution now :) Thanks you all for your contribution. Many regards, Bruno. On 17 Mar 2007 16:25:32 +0000, Bryan Henderson <bryanh@giraffe-data.com
wrote:
I have a solution.. but I think there can be a better one. My solution is: having a midlle program between syslog-ng and the destination file. And for every message received, the program would verify if the destination file (p.e. auth.log) already has 100 lines. If so, deleted the last line, make all others one ENTER down and that append the new line to the beginning of the file.
It would probably be better to maintain the file in circular fashion: allocate 8K in the beginning, then write messages from beginning to end, then wrap around and write from the beginning again, always keeping track (in a header of the file) of where you last wrote. Use a special program that understands this file format to translate it to a regular stream that you can use normally. Lots of systems have programs that do this. I looked on Freshmeat for a publicly available one and found 'cupyvei', though I don't much care for the details of this implementation. It would be easy to write a program of your own. I also think a built in log destination of this type (e.g. "file_circular") would be an excellent addition to syslog-ng. I think this type of logging is a common requirement. -- Bryan Henderson San Jose, California _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html