<div dir="ltr"><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Oct 2, 2014 at 9:33 PM, Jim Hendrick <span dir="ltr"><<a href="mailto:jrhendri@roadrunner.com" target="_blank">jrhendri@roadrunner.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi,<br>
<br>
I am working on configuring Elasticsearch, Logstash & Kibana (ELK) to<br>
test it as a backend search tool for large volumes of logs.<br>
<br>
I decided to put Redis in front of Logstash as a "broker" for the<br>
incoming logs, and syslog-ng as the "shipper" so it looks like this:<br>
<br>
syslog-ng ==> redis ==> logstash ==> elasticsearch ==> apache ==> kibana<br></blockquote><div><br></div><div>I've been using the following:<br><br>syslog-ng => rabbitmq => elasticsearch<br><br></div><div>syslog-ng + patterndb to parse logs and write then in json format on rabbitmq, after that is just use elasticsearch amqp river to consume the queue.<br></div></div><br></div></div>