[syslog-ng] Compressing syslog traffic across a WAN?

[K K]

Has anybody found a good way to take a high-volume stream of UDP
syslog packets, aggregate and compress the packets, and then recreate
them, with the original source IPs, at the other end of a WAN circuit?

Looks like Riverbed cannot optimize UDP syslog?.  Encapsulating the
packets in an IPSEC tunnel with compression would make the packets
smaller, but not reduce the packets-per-second.


Tolstoy Version:
I have a number of 'appliance' hosts on the west coast which generate
high volumes of syslog events, which I need to forward to an
'appliance' log analysis server (EIQ) in the midwest.  The version of
EIQ we're stuck with only supports UDP, and uses the source IP of the
packet to decide which host the event occurred on -- it cannot take
the embedded hostname in the packet and use that.

What we're doing right now is configuring all the west coast
appliances send their syslog events to a syslog-ng server locally on
the same network, which then filters out the junk events and uses
spoof-source to forward these UDP packets across the slow WAN circuit
to the EIQ appliances in the Midwest:

Source \
Source  --UDP-- syslog-ng --UDP-over-WAN--  EIQ
Source /

I could consider sending these events via TCP syslog to a syslog-ng
server in the Midwest, and that would then send a copy of the packets
via UDP to EIQ, but in this approach, I gather that we wouldn't be
able to use spoof-source to regenerate the UDP packet with the
original source IP?


Thanks,

Kevin