[syslog-ng] syslog-ng as "shipper" into ELK stack
jrhendri at roadrunner.com
Fri Oct 3 23:11:59 CEST 2014
Thanks! I will.
Sent from my Verizon Wireless 4G LTE smartphone
-------- Original message --------
From: Fabien Wernli <wernli at in2p3.fr>
Date:10/03/2014 4:12 AM (GMT-05:00)
To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu>
Subject: Re: [syslog-ng] syslog-ng as "shipper" into ELK stack
On Fri, Oct 03, 2014 at 12:33:41AM +0000, Jim Hendrick wrote:
> syslog-ng ==> redis ==> logstash ==> elasticsearch ==> apache ==> kibana
We've been using the following stack for over a year:
syslog-ng ==> logstash ==> elasticsearch
For various reasons, one being performance, we recently switched to:
syslog-ng ==> elasticsearch
This was done thanks to the syslog-ng-incubator perl module. I've set up a
small github repository where you can see our configuration .
> (I topped out today sending ~7000 events per second, and saw an insane
> amount of swapping going on)
I've had tremendous issues with LS when the workload was darting up.
Since we switched to perl, we still have issues, but they're certainly not
performance related: with a single perl destination we could easily keep up
10k events per second on a mediumish virtual machine.
> Is anyone aware of any plans to implement an elasticsearch destination?
The upcoming 3.6 version will ship with a "native" elasticsearch
destination, which currently however is only a wrapper script.
I'd highly appreciate if you could test a similar config to ours, in order
to share some experience.
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
-------------- next part --------------
An HTML attachment was scrubbed...
More information about the syslog-ng