[syslog-ng] Inline Compression

[Aaron Wiebe]

On Thu, Oct 2, 2008 at 4:06 AM, Balazs Scheidler <bazsi at balabit.hu> wrote:
>
> You could do something like this though:
>
> destination d_gzip { program("gzip -c >> logfile.gz"); };

While I see the advantages, I don't think the program handler works
with things like:

$YEAR
$MONTH
$DAY
$HOUR
$HOST

Now, obviously some of the date related items could be handled by
writing something that goes based on the local system time, but if
there are logs from remote systems that are slightly off in their
timestamps (ie, getting remote logs 5 seconds behind or something),
they'd end up in the wrong files.  At the same time, $HOST would
require parsing the logfile line - which could get messy.

Having it inline would make me really comfortable in using it.  I'm
guessing it was originally in the plan given the tags in the code for
a file option "compress(yes)"...  If I submitted a patch, would you
consider including it in a future release?

-Aaron

>
> This is somewhat better as gzip runs as a separate process and has time
> to close the logfile properly. Then you only need a cat program that
> understands that multiple gzip streams are concatenated. Hmmm, I've just
> checked simple zcat works.
>
> So as it seems this is much more robust to do with an external program
> than building it into syslog-ng, on the other hand this also means that
> you can use a 2nd CPU in the system for compression.
>
>
> --
> Bazsi
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>