[syslog-ng] Inline Compression

[Balazs Scheidler]

On Thu, 2008-10-02 at 11:35 -0400, Aaron Wiebe wrote:
> On Thu, Oct 2, 2008 at 4:06 AM, Balazs Scheidler <bazsi at balabit.hu> wrote:
> >
> > You could do something like this though:
> >
> > destination d_gzip { program("gzip -c >> logfile.gz"); };
> 
> While I see the advantages, I don't think the program handler works
> with things like:
> 
> $YEAR
> $MONTH
> $DAY
> $HOUR
> $HOST
> 
> Now, obviously some of the date related items could be handled by
> writing something that goes based on the local system time, but if
> there are logs from remote systems that are slightly off in their
> timestamps (ie, getting remote logs 5 seconds behind or something),
> they'd end up in the wrong files.  At the same time, $HOST would
> require parsing the logfile line - which could get messy.
> 
> Having it inline would make me really comfortable in using it.  I'm
> guessing it was originally in the plan given the tags in the code for
> a file option "compress(yes)"...  If I submitted a patch, would you
> consider including it in a future release?

Yes, I would, but please note the "Contributory License Agreement" on
our website.

On the technical side, I think doing it properly and robust enough, the
solution needs the "separate" process idea, otherwise you'd end up with
corrupt files whenever syslog-ng crashes.

So what I propose is to create a program() destination that can handle
macros, it would be useful for other purposes as well.

affile.c has code that you can pattern your implementation.

-- 
Bazsi