Thanks!  I will.   Jim Sent from my Verizon Wireless 4G LTE smartphone -------- Original message -------- From: Fabien Wernli <wernli@in2p3.fr> Date:10/03/2014 4:12 AM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng@lists.balabit.hu> Subject: Re: [syslog-ng] syslog-ng as "shipper" into ELK stack Hi Jim, On Fri, Oct 03, 2014 at 12:33:41AM +0000, Jim Hendrick wrote:

syslog-ng ==> redis ==> logstash ==> elasticsearch ==> apache ==> kibana

We've been using the following stack for over a year: syslog-ng ==> logstash ==> elasticsearch For various reasons, one being performance, we recently switched to: syslog-ng ==> elasticsearch This was done thanks to the syslog-ng-incubator perl module. I've set up a small github repository where you can see our configuration [1].

(I topped out today sending ~7000 events per second, and saw an insane amount of swapping going on)

I've had tremendous issues with LS when the workload was darting up. Since we switched to perl, we still have issues, but they're certainly not performance related: with a single perl destination we could easily keep up 10k events per second on a mediumish virtual machine.

Is anyone aware of any plans to implement an elasticsearch destination?

The upcoming 3.6 version will ship with a "native" elasticsearch destination, which currently however is only a wrapper script. I'd highly appreciate if you could test a similar config to ours, in order to share some experience. Cheers [1] https://github.com/faxm0dem/syslog_ng-elasticsearch ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq