[syslog-ng] syslog-ng as "shipper" into ELK stack

Jim Hendrick jrhendri at roadrunner.com
Sat Oct 4 04:09:57 CEST 2014

Thanks. Why rabbitmq  instead of redis?  Is it faster, or does it offer some additional  functions? 


Sent from my Verizon Wireless 4G LTE smartphone

-------- Original message --------
From: Alexandre Biancalana <biancalana at gmail.com> 
Date:10/03/2014  7:01 PM  (GMT-05:00) 
To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu> 
Subject: Re: [syslog-ng] syslog-ng as "shipper" into ELK stack 

On Thu, Oct 2, 2014 at 9:33 PM, Jim Hendrick <jrhendri at roadrunner.com> wrote:

   I am working on configuring Elasticsearch, Logstash & Kibana (ELK) to
test it as a backend search tool for large volumes of logs.

I decided to put Redis in front of Logstash as a "broker" for the
incoming logs, and syslog-ng as the "shipper" so it looks like this:

syslog-ng ==> redis ==> logstash ==> elasticsearch ==> apache ==> kibana

I've been using the following:

syslog-ng => rabbitmq => elasticsearch

syslog-ng + patterndb to parse logs and write then in json format on rabbitmq, after that is just use elasticsearch amqp river to consume the queue.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20141003/69c6c0ec/attachment-0001.htm 

More information about the syslog-ng mailing list