[syslog-ng] Inline Compression

[Balazs Scheidler]

On Wed, 2008-10-01 at 15:35 -0400, Aaron Wiebe wrote:
> Greetings,
> 
> I've noticed in 1.6.x that there are tags already built into the
> configuration file to support inline compression of file output  (but
> they complain if used).  Is there any plan to support this?  Are there
> reasons this hasn't been implemented yet?  Given my knowledge of the
> libgzip, I can't imagine its terribly hard.
> 
> The ability to write logs compressed would be -very- useful.

It is not so easy, as syslog-ng needs to reopen the same file multiple
times even if the previous shutdown was not clean. if syslog-ng crashed,
you still want to append logs to the same file. This cannot be cleanly
done with libz alone. And the requirement that the same file cannot be
written to any time again is too much restriction.

You could do something like this though:

destination d_gzip { program("gzip -c >> logfile.gz"); };

This is somewhat better as gzip runs as a separate process and has time
to close the logfile properly. Then you only need a cat program that
understands that multiple gzip streams are concatenated. Hmmm, I've just
checked simple zcat works.

So as it seems this is much more robust to do with an external program
than building it into syslog-ng, on the other hand this also means that 
you can use a 2nd CPU in the system for compression.


-- 
Bazsi