I was thinking more of the Elastic, sumo and splunk way where the messages are buffered compressed and sent using some standard open source compression library. On Wed, Mar 14, 2018 at 7:41 AM, Gergely Nagy <algernon@balabit.com> wrote:
"Scot" == Scot <scotrn@gmail.com> writes:
Scot> We have 2 syslog relays that send data over strait TCP right now to another Scot> syslog-ng master. Scot> I was looking for ways to optimize that communication as well as a network Scot> JSON input from logstash and how other connections to the above could be Scot> optimized.
I think TLS compression is a viable, practical solution in this case. It's easy to set up, and as the compression applies to the whole stream, and isn't done on a per-message basis (which would be horribly inefficient), it has the potential of achieving compression ratios that offset the overhead of TLS.
The alternatives (like using a custom program destination and source) are - in my opinion - considerably harder to set up well, because you'd need to figure out a way to get the compressed stuff from one host to another, and the tcp()/udp()/syslog() methods don't work here, because they aren't well suited for transfer of binary data, and generally operate on a per-message basis, while you'd rather send a continuous stream.
-- |8] ____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq