On Mon, 2010-09-27 at 09:38 -0500, Martin Holste wrote:
That could definitely be helpful, but I think the big one I'd be looking for would be something more basic that fires for a log chain when the rollover occurs, such as:
destination d_file { file("/var/log/messages.$MIN" events( on_rotate( exec("/usr/local/bin/messages-file-finished") ) ) ) };
I'm sure you could replicate this by using your example syntax and making sure that your time macro in the destination file name matches the pseudo-cron entry, but it seems like that might introduce some small issues with synchronization or race conditions.
Yes. The problem with this, that there's no such thing as rollover. :( syslog-ng keeps expanding the template string to find out which files to write and then times out files that do not get written to after time_reap() seconds. The issue is that files may get closed even in the middle of the minute. So the only sane way to react to "rollovers" is based on time. ... and there's also an issue with nonsynchronized clocks, so the same file can be written to when the local time is way past that, therefore you have to calculate with which timestamps to trust. In our SSB product for example we tend to use the local time now (e.g. R_DATE) because of this reason.
On Mon, Sep 27, 2010 at 9:24 AM, Balazs Scheidler <bazsi@balabit.hu> wrote:
On Mon, 2010-09-20 at 14:35 -0500, Martin Holste wrote:
commit 70e91556b6af8724334443347fd6488745405344 Author: Balazs Scheidler <bazsi@balabit.hu> Date: Mon Sep 20 17:12:27 2010 +0200
convertfuncs: new plugin to contain conversion template functions
The plugin now only contains ipv4-to-int which converts an IPv4 address to a long integer.
Usage:
$(ipv4-to-int $SOURCEIP)
Very cool stuff!
Expect a blog post on this topic, a simple correllation engine is now built into patterndb.
Hm, very interesting, I'll be taking a look.
Regarding detecting the batches being complete: It seems a little inelegant to have a baby-sitter script that looks for an appropriately named file in a given directory and hoping it's the right buffer. It would be really nice if Syslog-NG could execute program() on a file that has just been written to for the last time.
I was thinking about adding "events" to sources/destinations which could invoke 3rd party tools/scripts when something happens.
Events could be time based, but other setup/teardown style stuff can come in handy.
e.g.
destination d_file { file("/var/log/messages.$HOUR" events(cron(min(5) hour(*) exec("/usr/local/bin/messages-file-finished")); };
Not sure about the syntax though. Also I want it to be able to run processes like tail -f:
source s_follow { pipe("/var/run/syslog-ng/tail-pipe" events(startup(supervise("/usr/bin/tail -f /var/log/apache.log > /var/run/syslog-ng/tail-pipe")))); };
I know that syslog-ng is capable for tailing files, but the point is that there are sometimes complex log systems of various applications, and the only sane interface to them to run a process to tail its otherwise binary logfile. I want syslog-ng to manage these processes.
-- Bazsi
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html
-- Bazsi