Logrotate Questions And Solicitation For Ideas...
Hello All; I've got a syslog-1.6.9 server running quite well with phpsyslog-ng. I am trying to rotate my logs and maintain just 14 days worth of logs within my MySql DB, an issue that I am having and will hopefully get fixed within the phpSyslog-NG group. With regards to flat file logs, the issue that I am writing here for, I wish to have 30 days worth of data within my log files (flat files under /var/log). The problem that I am having is rotating under syslog-ng, *NOT* MySql. Can anyone please provide me with a good method for rotating logs, where my logs are kept under various subdirs, depending on source address? Attached is my syslog-ng.conf, in case anyone cares to read it to determine what my dir-layouts are, also I am looking for help on why some of my source hosts are getting into two locations, I am sure that it is bacause of the "filter" rules in my config. Does the below make sense? # Trying to make all hosts that start with sw or contain sw1, or sw2, # get placed where filter F_switch should go. filter F_switch { host("sw*") or host("sw1") or host("sw2"); }; # Trying to place logs for all hosts that start with fw to go to where filter F_firewall should go. filter F_firewall { host("^fw*"); }; Please, I am sorry for this question if it is in the FAQ, I am having problem finding this info. Thank you all! .vp
I am using BigBrother to monitor a group of machines and I am very intersted in knowing if anyone has gotten syslog-ng to alert to bigbrother based on message severity? If so, could you please describe what was done in order to make this work? Thank you very much. .vp
Does anyone have an answer for this question ? I hadnt seen any responses yet. Thank you -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Vadim Pushkin Sent: Wednesday, July 19, 2006 2:43 PM To: syslog-ng@lists.balabit.hu Subject: [syslog-ng] Syslog-NG to BigBrother Anyone? I am using BigBrother to monitor a group of machines and I am very intersted in knowing if anyone has gotten syslog-ng to alert to bigbrother based on message severity? If so, could you please describe what was done in order to make this work? Thank you very much. .vp _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
On Wed, 19 Jul 2006 18:34:25 -0000, Vadim Pushkin said:
The problem that I am having is rotating under syslog-ng, *NOT* MySql. Can anyone please provide me with a good method for rotating logs, where my logs are kept under various subdirs, depending on source address?
What I do is something along the lines of: 1) destination foo { file("/var/logs/whatever/messages-$MONTH$DAY"); }; so syslog-ng creates a seperate output file for each day. 2) A cronjob that then does 'find /var/logs/whatever -mtime +30 -exec rm {}\;' is all that's needed to clean up. Add salt, pepper, and documentation to taste.
Hi Valdis! The method you recommend would completely wipe out my logs, I wish to rotate them and keep at least N days worth. .vp PS Does anyone know if I must restart syslog-ng after or before a logrotate?
From: Valdis.Kletnieks@vt.edu
On Wed, 19 Jul 2006 18:34:25 -0000, Vadim Pushkin said:
The problem that I am having is rotating under syslog-ng, *NOT* MySql. Can anyone please provide me with a good method for rotating logs, where my logs are kept under various subdirs, depending on source address?
What I do is something along the lines of:
1) destination foo { file("/var/logs/whatever/messages-$MONTH$DAY"); };
so syslog-ng creates a seperate output file for each day.
2) A cronjob that then does 'find /var/logs/whatever -mtime +30 -exec rm {}\;' is all that's needed to clean up. Add salt, pepper, and documentation to taste.
On Wed, 19 Jul 2006 20:51:43 -0000, Vadim Pushkin said:
Hi Valdis!
The method you recommend would completely wipe out my logs, I wish to rotate them and keep at least N days worth.
Notice the "-mtime +30" - that means only do files that have not been changed in 30 days.....
Again, I am looking to rotate my logs, not delete them. Ideally, something like retaining *.logs.1 *.logs.2, etc. .vp
From: Valdis.Kletnieks@vt.edu
On Wed, 19 Jul 2006 20:51:43 -0000, Vadim Pushkin said:
Hi Valdis!
The method you recommend would completely wipe out my logs, I wish to rotate them and keep at least N days worth.
Notice the "-mtime +30" - that means only do files that have not been changed in 30 days.....
On Wed, 19 Jul 2006 22:26:15 -0000, Vadim Pushkin said:
Again, I am looking to rotate my logs, not delete them. Ideally, something like retaining *.logs.1 *.logs.2, etc.
Oh my. They *have* to be called logs, logs.1, logs.2, logs.3... logs.30? :) We find it a lot easier if they're called: [/logs/aggregate/2006/07]1 ls maillog-2006-0701 maillog-2006-0711 messages-2006-0702 messages-2006-0712 maillog-2006-0702 maillog-2006-0712 messages-2006-0703 messages-2006-0713 maillog-2006-0703 maillog-2006-0713 messages-2006-0704 messages-2006-0714 maillog-2006-0704 maillog-2006-0714 messages-2006-0705 messages-2006-0715 maillog-2006-0705 maillog-2006-0715 messages-2006-0706 messages-2006-0716 maillog-2006-0706 maillog-2006-0716 messages-2006-0707 messages-2006-0717 maillog-2006-0707 maillog-2006-0717 messages-2006-0708 messages-2006-0718 maillog-2006-0708 maillog-2006-0718 messages-2006-0709 messages-2006-0719 maillog-2006-0709 maillog-2006-0719 messages-2006-0710 maillog-2006-0710 messages-2006-0701 messages-2006-0711 (there's more in ../06 and ../05, going back some 90 days actually). It's a lot easier that way than trying to remember how many days ago June 28 was - we *know* the msgs are in 06/*0628. That's especially helpful when you're trying to troubleshoot something at 1AM and keep forgetting it's another day back and keep wondering why you can't find it in June 29th's logs. ;)
On Wed, Jul 19, 2006 at 10:07:39PM -0400, Valdis.Kletnieks@vt.edu wrote:
On Wed, 19 Jul 2006 22:26:15 -0000, Vadim Pushkin said:
Again, I am looking to rotate my logs, not delete them. Ideally, something like retaining *.logs.1 *.logs.2, etc.
Oh my. They *have* to be called logs, logs.1, logs.2, logs.3... logs.30? :)
Go with what Valdis recommends. If you really have to rotate them, use logrotate. -- Nate "My way of joking is to tell the truth. That's the funniest joke in the world." - Muhammad Ali
Hi Nate; I'd love to use logrotate, but as you can see from my original post, my logs are in a bunch of dirs, mostly created on the fly as new stuff comes in. How do I configure logrotate.conf for this? Thank you, .vp
From: Nate Campi <nate@campin.net>
On Wed, Jul 19, 2006 at 10:07:39PM -0400, Valdis.Kletnieks@vt.edu wrote:
On Wed, 19 Jul 2006 22:26:15 -0000, Vadim Pushkin said:
Again, I am looking to rotate my logs, not delete them. Ideally, something like retaining *.logs.1 *.logs.2, etc.
Oh my. They *have* to be called logs, logs.1, logs.2, logs.3... logs.30? :)
Go with what Valdis recommends. If you really have to rotate them, use logrotate. -- Nate
I believe this will work... I haven't actually tested it though ;-) create a file in logrotate.d called <blah> Its content would look like: /var/log/somedir/*/*log { missingok notifempty sharedscripts postrotate /bin/kill -HUP `cat /var/run/syslog.pid 2>/dev/null` 2> /dev/null || true endscript } HTH, Harry -- Harry Hoffman Integrated Portable Solutions, LLC 877.846.5927 ext 1000 http://www.ip-solutions.net/ Vadim Pushkin wrote:
Hi Nate;
I'd love to use logrotate, but as you can see from my original post, my logs are in a bunch of dirs, mostly created on the fly as new stuff comes in. How do I configure logrotate.conf for this?
Thank you,
.vp
From: Nate Campi <nate@campin.net>
On Wed, Jul 19, 2006 at 10:07:39PM -0400, Valdis.Kletnieks@vt.edu wrote:
On Wed, 19 Jul 2006 22:26:15 -0000, Vadim Pushkin said:
Again, I am looking to rotate my logs, not delete them. Ideally, something like retaining *.logs.1 *.logs.2, etc.
Oh my. They *have* to be called logs, logs.1, logs.2, logs.3... logs.30? :)
Go with what Valdis recommends. If you really have to rotate them, use logrotate. -- Nate
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
Vadim, in syslog_ng.conf: destination autorotatedlogs { file( "/var/logs/$HOST/$HOST.$DAY.log" remove_if_older(2591000) ); }; This should create 'server01-.04.log', and delete that file if it's older than 30 days (minus 1000 seconds for possible skewing), and start fresh anew the next time the 4th day of the month rolls by. This should set you up with a 30 day log rotation. -JP Senior jpsenior@veer.com 403-313-5514 -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Vadim Pushkin Sent: Thursday, July 20, 2006 12:53 PM To: syslog-ng@lists.balabit.hu Subject: Re: [syslog-ng] Logrotate Questions And Solicitation For Ideas... Hi Nate; I'd love to use logrotate, but as you can see from my original post, my logs are in a bunch of dirs, mostly created on the fly as new stuff comes in. How do I configure logrotate.conf for this? Thank you, .vp
From: Nate Campi <nate@campin.net>
On Wed, Jul 19, 2006 at 10:07:39PM -0400, Valdis.Kletnieks@vt.edu wrote:
On Wed, 19 Jul 2006 22:26:15 -0000, Vadim Pushkin said:
Again, I am looking to rotate my logs, not delete them. Ideally, something like retaining *.logs.1 *.logs.2, etc.
Oh my. They *have* to be called logs, logs.1, logs.2, logs.3... logs.30? :)
Go with what Valdis recommends. If you really have to rotate them, use logrotate. -- Nate
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
I wish to rotate them and keep at least N days worth.
I believe the OP is thinking the term "rotate" differently than others here. If the desired effect is to always have today's events and 29 days into the past,it's much easier to to have syslog-ng create and write to logfiles named by the day-of-month, no manual "rotation" necessary. On 7/20/06, JP Senior <JPSenior@veer.com> wrote:
Vadim, in syslog_ng.conf:
destination autorotatedlogs { file( "/var/logs/$HOST/$HOST.$DAY.log" remove_if_older(2591000) ); };
This should create 'server01-.04.log', and delete that file if it's older than 30 days (minus 1000 seconds for possible skewing), and start fresh anew the next time the 4th day of the month rolls by.
This is what I do, it works very well, I actually log by hour of the day, so I use: "/var/logs/$HOST/$DAY/$HOUR.log"
This should set you up with a 30 day log rotation.
Sometimes you will end up with a few old logfiles left sitting around, for example, in months with fewer than 31 days. If this is a real problem you can add a cron script to explicitly purge logs older than 30 days, as mentioned earlier in this thread. Kevin
participants (7)
-
Harry Hoffman
-
JP Senior
-
Kevin
-
Nate Campi
-
Stuart Boyle
-
Vadim Pushkin
-
Valdis.Kletnieks＠vt.edu