<div dir="ltr">logstash-* index is for logs that have been ingested via logstash of course :) <div><br></div><div>every component of ELK scales horizontally extremely well. </div></div><div id="DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2"><br>
<table style="border-top:1px solid #aaabb6">
<tr>
<td style="width:55px;padding-top:13px"><a href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=icon" target="_blank"><img src="https://ipmcdn.avast.com/images/2016/icons/icon-envelope-tick-round-orange-v1.png"></a></td>
<td style="width:470px;padding-top:15px;color:#41424e;font-size:13px;font-family:Arial,Helvetica,sans-serif;line-height:18px">Virus-free. <a href="https://www.avast.com/sig-email?utm_medium=email&utm_source=link&utm_campaign=sig-email&utm_content=webmail&utm_term=link" target="_blank" style="color:#4453ea">www.avast.com</a>
</td>
</tr>
</table><a href="#DDB4FAA8-2DD7-40BB-A1B8-4E2AA1F9FDF2" width="1" height="1"></a></div><div class="gmail_extra"><br><div class="gmail_quote">On Wed, Apr 20, 2016 at 12:41 PM, Scot Needy <span dir="ltr"><<a href="mailto:scotrn@gmail.com" target="_blank">scotrn@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div><br></div><div><div><a href="https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=kibana%20dashboard%20template" target="_blank">https://www.google.com/webhp?sourceid=chrome-instant&ion=1&espv=2&ie=UTF-8#q=kibana%20dashboard%20template</a></div></div><div><br></div><div>May have misspoke. Using ELK and patterndb.xml is new to me and I am still trying to learn the mechanics. </div><div><br></div><div><br></div><div> I started by looking at Google for Kibana dashboard templates, one of the better results here. </div><div><a href="https://github.com/markwalkom/kibana-dashboards" target="_blank">https://github.com/markwalkom/kibana-dashboards</a> Most of the kibana json templates I have seen on the net are setup for a logstash-* “index” ?. </div><div><br></div><div>I’m trying to set Syslog-ng-> ELK up in my “spare time” at work. So time and ease of setup and support community size are big considerations. I want to enable GeoIP for ASA data, NetFlow data and be able to leverage existing templates logstash or patterndb for common applications. Apache, Linux Syslog, Storage syslog, etc… </div><div><div class="h5"><div><br></div><div><br></div><br><div><blockquote type="cite"><div>On Apr 20, 2016, at 2:13 PM, Scheidler, Balázs <<a href="mailto:balazs.scheidler@balabit.com" target="_blank">balazs.scheidler@balabit.com</a>> wrote:</div><br><div><p dir="ltr">Can you pls point me to the direction of the logstash material you mentioned? I would be interested in them whether it'd be possible to port them over.</p>
<div class="gmail_quote">On Apr 20, 2016 7:00 PM, "Scot Needy" <<a href="mailto:scotrn@gmail.com" target="_blank">scotrn@gmail.com</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div style="word-wrap:break-word"><div>Some thoughts on my deployment</div><div><br></div><div><b>Logstash</b></div><div>I think I’m going to need to re-introduce logstash just to leverage the existing open source material of logstash filters and Kibana desktops. </div><div>VMware, ASA for example but wanted more real time data. I could probably do the realtime tags with pattendb. </div><div> </div><div><b>syslog-ng counters</b> </div><div>We use an IPAM API to create unique filters, log and destination conf files. The goal was to get unique syslog counters for every VLAN realtime directly from syslog-ng-ctl stats.. </div><div><br></div><div><br></div><div>@include IPAM-filters</div><div><div>filter f_192_168_252_0 { netmask(<a href="http://192.168.252.0/24);" target="_blank">192.168.252.0/24);</a>};</div><div>filter f_192_168_253_0 { netmask(<a href="http://192.168.253.0/24);" target="_blank">192.168.253.0/24);</a>};</div><div>filter f_192_168_254_0 { netmask(<a href="http://192.168.254.0/30);" target="_blank">192.168.254.0/30);</a>};</div></div><div><br></div><div><br></div><div>@include IPAM-dest.conf</div><div><div>destination d_192_168_252_0 { file(/opt/syslog-ng/logs/192_168_252_0/$YEAR$MONTH$DAY-$HOUR-$HOST.log);};</div><div>destination d_192_168_253_0 { file(/opt/syslog-ng/logs/192_168_253_0/$YEAR$MONTH$DAY-$HOUR-$HOST.log);};</div><div>destination d_192_168_254_0 { file(/opt/syslog-ng/logs/192_168_254_0/$YEAR$MONTH$DAY-$HOUR-$HOST.log);};</div></div><div><br></div><div>@include IPAM-log.conf</div><div><div>log { source(s_net); filter(f_192_168_252_0); destination(d_192_168_252_0);};</div><div>log { source(s_net); filter(f_192_168_253_0); destination(d_192_168_253_0);};</div><div>log { source(s_net); filter(f_192_168_254_0); destination(d_192_168_254_0);};</div><div>log { source(s_net); filter(f_192_168_254_4); destination(d_192_168_254_4);};</div></div><div><br></div><div><br></div><br><div><blockquote type="cite"><div>On Apr 20, 2016, at 11:18 AM, Scot Needy <<a href="mailto:scotrn@gmail.com" target="_blank">scotrn@gmail.com</a>> wrote:</div><br><div><br><br>Hi, <br><br> Does anyone have links or care to share notes on making a syslog-ng -> ELK scale for enterprise ? <br><br>I have some ideas and will gladly share my solution but also don’t want to spend days figuring these things out that have already been built. <br>There are many ELK specific references but I also want to make sure the model fits the syslog workload. <br><br><br>Thanks <br><br></div></blockquote></div><br></div><br>______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
<br></blockquote></div>
______________________________________________________________________________<br>Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br><br></div></blockquote></div><br></div></div></div><br>______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/syslog-ng-faq</a><br>
<br>
<br></blockquote></div><br></div>