[syslog-ng] Compression on input and output ?

[Scot]

I was thinking more of the Elastic, sumo and splunk way where the messages
are buffered compressed and sent using some standard open source
compression library.



On Wed, Mar 14, 2018 at 7:41 AM, Gergely Nagy <algernon at balabit.com> wrote:

> >>>>> "Scot" == Scot  <scotrn at gmail.com> writes:
>
>     Scot> We have 2 syslog relays that send data over strait TCP right now
> to another
>     Scot> syslog-ng master.
>     Scot> I was looking for ways to optimize that communication as well as
> a network
>     Scot> JSON input from logstash and how other connections to the above
> could be
>     Scot> optimized.
>
> I think TLS compression is a viable, practical solution in this case.
> It's easy to set up, and as the compression applies to the whole stream,
> and isn't done on a per-message basis (which would be horribly
> inefficient), it has the potential of achieving compression ratios that
> offset the overhead of TLS.
>
> The alternatives (like using a custom program destination and source)
> are - in my opinion - considerably harder to set up well, because you'd
> need to figure out a way to get the compressed stuff from one host to
> another, and the tcp()/udp()/syslog() methods don't work here, because
> they aren't well suited for transfer of binary data, and generally
> operate on a per-message basis, while you'd rather send a continuous
> stream.
>
> --
> |8]
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180314/5d63f1cf/attachment.html>