Re: [syslog-ng] syslog-ng as "shipper" into ELK stack

On Thu, Oct 2, 2014 at 9:33 PM, Jim Hendrick <jrhendri@roadrunner.com> wrote:

Hi,

I am working on configuring Elasticsearch, Logstash & Kibana (ELK) to test it as a backend search tool for large volumes of logs.

I decided to put Redis in front of Logstash as a "broker" for the incoming logs, and syslog-ng as the "shipper" so it looks like this:

syslog-ng ==> redis ==> logstash ==> elasticsearch ==> apache ==> kibana

I've been using the following: syslog-ng => rabbitmq => elasticsearch syslog-ng + patterndb to parse logs and write then in json format on rabbitmq, after that is just use elasticsearch amqp river to consume the queue.