Hi, K K <kkadow@gmail.com> [20070322 14:02:27 -0500]:
Has anybody found a good way to take a high-volume stream of UDP syslog packets, aggregate and compress the packets, and then recreate them, with the original source IPs, at the other end of a WAN circuit?
Looks like Riverbed cannot optimize UDP syslog?. Encapsulating the packets in an IPSEC tunnel with compression would make the packets smaller, but not reduce the packets-per-second.
This would be a 'trivial' thing to do with a Perl script....it would not even have to decode the packets, just send and relay the messages at the other end. A short term hack would be to use ppp (load on the compression) with netcat over TCP. A quick search over at freshmeat.net (should always be a first port of call for people) is: http://www.winton.org.uk/zebedee/manual.html Cheers Alex
Tolstoy Version: I have a number of 'appliance' hosts on the west coast which generate high volumes of syslog events, which I need to forward to an 'appliance' log analysis server (EIQ) in the midwest. The version of EIQ we're stuck with only supports UDP, and uses the source IP of the packet to decide which host the event occurred on -- it cannot take the embedded hostname in the packet and use that.
What we're doing right now is configuring all the west coast appliances send their syslog events to a syslog-ng server locally on the same network, which then filters out the junk events and uses spoof-source to forward these UDP packets across the slow WAN circuit to the EIQ appliances in the Midwest:
Source \ Source --UDP-- syslog-ng --UDP-over-WAN-- EIQ Source /
I could consider sending these events via TCP syslog to a syslog-ng server in the Midwest, and that would then send a copy of the packets via UDP to EIQ, but in this approach, I gather that we wouldn't be able to use spoof-source to regenerate the UDP packet with the original source IP?
Thanks,
Kevin _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html