From the description in the reference manual I was expecting remove_if_older() to cause a destination file to be deleted once the file becomes older than the specified age, in seconds. However, this is not the behaviour I have observed. It seems that the age check is only made the first time the file is written too. I've had a quick look through the code and it looks like the age check is only made in affile_dw_init, which I assume is called when the file is first opened. This would be consistent with my observations.
Is this the intended behaviour? Thanks Anthony
On Wed, 20 Dec 2006 09:52:05 +1300, anthony lineham said:
From the description in the reference manual I was expecting remove_if_older() to cause a destination file to be deleted once the file becomes older than the specified age, in seconds. However, this is not the behaviour I have observed. It seems that the age check is only made the first time the file is written too. I've had a quick look through the code and it looks like the age check is only made in affile_dw_init, which I assume is called when the file is first opened. This would be consistent with my observations.
Is this the intended behaviour?
If it wasn't, you'd have the behavior of "nuke the logfile if nobody's logged to it in the last N seconds". Probably not the behavior you want, because it would just suck. Consider a server that spews a "I'm crashing because XYZ failed" message, and then shuts up for 12 hours because it hasn't been restarted. You go to look for the message - and discover that the file evaporated because it was older than 12*3600 seconds. Whoops. :) (Yes, the case I'm making up is probably a misconfig on multiple grounds, but it demonstrates the sort of counter-intuitive behavior you'd get...) Even more importantly - if the file is older than N seconds, it's because no messages have *gone* there for N seconds. This means that to get rid of a 12-hour-unwritten log, you need a new event handler that sets timer events and removes the file even when there *isn't* traffic - greatly adding to the code complexity. A tree falling in the forest *will* make a sound, if a requirement for it to fall is that somebody stop by with an axe....
<Valdis.Kletnieks@vt.edu> 20/12/2006 10:39 a.m. >>> On Wed, 20 Dec 2006 09:52:05 +1300, anthony lineham said: From the description in the reference manual I was expecting remove_if_older() to cause a destination file to be deleted once the file becomes older than the specified age, in seconds. However, this is not the behaviour I have observed. It seems that the age check is only made the first time the file is written too. I've had a quick look through the code and it looks like the age check is only made in affile_dw_init, which I assume is called when the file is first opened. This would be consistent with my observations.
Is this the intended behaviour?
If it wasn't, you'd have the behavior of "nuke the logfile if nobody's logged to it in the last N seconds". Probably not the behavior you want, because it would just suck. Consider a server that spews a "I'm crashing because XYZ failed" message, and then shuts up for 12 hours because it hasn't been restarted. You go to look for the message - and discover that the file evaporated because it was older than 12*3600 seconds. Whoops. :)
(Yes, the case I'm making up is probably a misconfig on multiple grounds, but it demonstrates the sort of counter-intuitive behavior you'd get...)
Even more importantly - if the file is older than N seconds, it's because no messages have *gone* there for N seconds. This means that to get rid of a 12-hour-unwritten log, you need a new event handler that sets timer events and removes the file even when there *isn't* traffic - greatly adding to the code complexity. A tree falling in the forest *will* make a sound, if a requirement for it to fall is that somebody stop by with an axe....
Ok, that makes a lot of sense. Although it does make me wonder what the intended use of this option actually is. This situation I'm working with is that I have a log file that is receiving status messages periodically. The messages are incremental and I'm only interested in the latest one. The advertised behaviour is that it checks the age before wirting a new message. So, you can imagine the prospect of being able to get the file to quietly rotate itself and always having the latest info was quite appealing.
It seems that the age check is only made the first time the file is written too. I've had a quick look through the code and it looks like the age check is only made in affile_dw_init, which I assume is called when the file is first opened. This would be consistent with my observations.
Is this the intended behaviour?
I believe it is called when first opened, and also when existing files are re-opened?
Ok, that makes a lot of sense. Although it does make me wonder what the intended use of this option actually is.
I believe the intended use is this: options { time_reap(3600); . . . } . . . destination d_routers { file("/var/log/routers/$WEEKDAY/$HOUR" remove_if_older(259200) ); } In this case, logs are automatically "rotated", by virtue of being written to directories named for the day of week, and files named for the hour of the day. So right now syslog-ng is writing /var/log/routers/Tue/16. When next Tuesday comes around, the old "16" file should be removed and recreated, instead of appending to week-old data. Kevin
Thanks, that helps. So in theory through careful co-ordination of time_reap, remove_if_older and the generation of the status message I'm interested in, I should be able to achieve what I want. Anthony
"K K" <kkadow@gmail.com> 12/20/06 11:55 AM >>> It seems that the age check is only made the first time the file is written too. I've had a quick look through the code and it looks like the age check is only made in affile_dw_init, which I assume is called when the file is first opened. This would be consistent with my observations.
Is this the intended behaviour?
I believe it is called when first opened, and also when existing files are re- opened?
Ok, that makes a lot of sense. Although it does make me wonder what the intended use of this option actually is.
I believe the intended use is this: options { time_reap(3600); . . . } . . . destination d_routers { file("/var/log/routers/$WEEKDAY/$HOUR" remove_if_older(259200) ); } In this case, logs are automatically "rotated", by virtue of being written to directories named for the day of week, and files named for the hour of the day. So right now syslog- ng is writing /var/log/routers/Tue/16. When next Tuesday comes around, the old "16" file should be removed and recreated, instead of appending to week- old data. Kevin _______________________________________________ syslog- ng maillist - syslog- ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog- ng Frequently asked questions at http://www.campin.net/syslog- ng/faq.html
On Wed, 2006-12-20 at 12:34 +1300, anthony lineham wrote:
Thanks, that helps. So in theory through careful co-ordination of time_reap, remove_if_older and the generation of the status message I'm interested in, I should be able to achieve what I want.
And if you are there, I'd appreciate some help in naming the "remove_if_older" option. As you might have experienced it is sometimes difficult to graps, but I could not come up with anything else (not being a native English speaker does not help either) -- Bazsi
I think the name is quite reasonable as it is. However you should upgrade the documentation to say that the check is only made when the file is first opened, not when each message is written. You should also upgrade the documentation for time_reap() as it seems that this only applies to files which have macros in the name. Regards Anthony
Balazs Scheidler <bazsi@balabit.hu> 21/12/2006 1:04 a.m. >>> On Wed, 2006-12-20 at 12:34 +1300, anthony lineham wrote: Thanks, that helps. So in theory through careful co-ordination of time_reap, remove_if_older and the generation of the status message I'm interested in, I should be able to achieve what I want.
And if you are there, I'd appreciate some help in naming the "remove_if_older" option. As you might have experienced it is sometimes difficult to graps, but I could not come up with anything else (not being a native English speaker does not help either)
-- Bazsi
On Thu, 2006-12-21 at 12:22 +1300, anthony lineham wrote:
I think the name is quite reasonable as it is. However you should upgrade the documentation to say that the check is only made when the file is first opened, not when each message is written.
I applied this patch to the docs: <row> <entry>remove_if_older()</entry> <entry>number</entry> - <entry>If set to a value higher than 0, before writing to a file, - syslog-ng checks whether this file is older than the specified + <entry>If set to a value higher than 0, before reopening a file, + syslog-ng checks whether the destination file is older than the specified amount of time (specified in seconds). If so, it removes the existing file and the line to be written is the first line of a new file having the same name. In combination with e.g.: the Is this easier to understand now?
You should also upgrade the documentation for time_reap() as it seems that this only applies to files which have macros in the name.
Which is true. -- Bazsi
Yes, that is clear now. Thanks Anthony
Balazs Scheidler <bazsi@balabit.hu> 21/12/2006 9:21 p.m. >>> On Thu, 2006-12-21 at 12:22 +1300, anthony lineham wrote: I think the name is quite reasonable as it is. However you should upgrade the documentation to say that the check is only made when the file is first opened, not when each message is written.
I applied this patch to the docs: <row> <entry>remove_if_older()</entry> <entry>number</entry> - <entry>If set to a value higher than 0, before writing to a file, - syslog-ng checks whether this file is older than the specified + <entry>If set to a value higher than 0, before reopening a file, + syslog-ng checks whether the destination file is older than the specified amount of time (specified in seconds). If so, it removes the existing file and the line to be written is the first line of a new file having the same name. In combination with e.g.: the Is this easier to understand now?
You should also upgrade the documentation for time_reap() as it
seems
that this only applies to files which have macros in the name.
Which is true. -- Bazsi _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
participants (4)
-
anthony lineham
-
Balazs Scheidler
-
K K
-
Valdis.Kletnieks＠vt.edu