[syslog-ng] Buffering AF_UNIX Destination, Batch Post Processing Messages

[Martin Holste]

That could definitely be helpful, but I think the big one I'd be
looking for would be something more basic that fires for a log chain
when the rollover occurs, such as:

destination d_file { file("/var/log/messages.$MIN" events( on_rotate(
exec("/usr/local/bin/messages-file-finished") ) ) ) };

I'm sure you could replicate this by using your example syntax and
making sure that your time macro in the destination file name matches
the pseudo-cron entry, but it seems like that might introduce some
small issues with synchronization or race conditions.

On Mon, Sep 27, 2010 at 9:24 AM, Balazs Scheidler <bazsi at balabit.hu> wrote:
> On Mon, 2010-09-20 at 14:35 -0500, Martin Holste wrote:
>> > commit 70e91556b6af8724334443347fd6488745405344
>> > Author: Balazs Scheidler <bazsi at balabit.hu>
>> > Date:   Mon Sep 20 17:12:27 2010 +0200
>> >
>> >    convertfuncs: new plugin to contain conversion template functions
>> >
>> >    The plugin now only contains ipv4-to-int which converts an IPv4 address
>> >    to a long integer.
>> >
>> > Usage:
>> >
>> > $(ipv4-to-int $SOURCEIP)
>> >
>>
>> Very cool stuff!
>>
>> > Expect a  blog post on this topic, a simple correllation engine is now
>> > built into patterndb.
>>
>> Hm, very interesting, I'll be taking a look.
>>
>> Regarding detecting the batches being complete:  It seems a little
>> inelegant to have a baby-sitter script that looks for an appropriately
>> named file in a given directory and hoping it's the right buffer.  It
>> would be really nice if Syslog-NG could execute program() on a file
>> that has just been written to for the last time.
>
> I was thinking about adding "events" to sources/destinations which could
> invoke 3rd party tools/scripts when something happens.
>
> Events could be time based, but other setup/teardown style stuff can
> come in handy.
>
> e.g.
>
> destination d_file { file("/var/log/messages.$HOUR"
>                        events(cron(min(5) hour(*) exec("/usr/local/bin/messages-file-finished"));
> };
>
> Not sure about the syntax though. Also I want it to be able to run processes like tail -f:
>
> source s_follow { pipe("/var/run/syslog-ng/tail-pipe"
>                        events(startup(supervise("/usr/bin/tail -f /var/log/apache.log > /var/run/syslog-ng/tail-pipe"))));
> };
>
> I know that syslog-ng is capable for tailing files, but the point is that there
> are sometimes complex log systems of various applications, and the only sane interface
> to them to run a process to tail its otherwise binary logfile. I want syslog-ng to
> manage these processes.
>
>
> --
> Bazsi
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>