<div dir="ltr"><div><div><div><div><div><div><div>Hi<br></div> Martinez<br></div> You need to install syslog-ng first. Then create configuration file for your linux and BSD machines inside /etc/syslog-ng/conf.d<br><br></div>Say you have configured one linux machine that forwards logs to this centralized syslog-ng server then you can use following configuration so that logs gets transferred to kibana.<br><br></div>Forwarder (Linux machine) -> Syslog-ng ( Centralize log collector) -> Elastic search -> Kibana<br><br></div>One such example can be like.<br><br></div># vi /etc/syslog-ng/conf.d/remote-linux-1.conf<br><br>
<p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify">
source s_2514 { tcp(port(2514)); }; <br></p><p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify"># Assuming remote linux machine forwarding logs to syslog-ng server's tcp port 2514.<br></p>
<p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify">
</p>
<p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify">
</p>
<p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify">
destination d_remote_linux1 { tcp("127.0.0.1" port(9200)
template("$(format-json --scope selected_macros --scope
nv_pairs)\n")); };</p><p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify"># Change the above IP -> 127.0.0.1 to your elastic server's IP<br></p>
<p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify">
</p><p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify">
filter f_remote_linux1 { '' your filter " ; };</p><p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify"># Apply whatever filter you want, you can use multiple filters as well.<br></p><p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify">
log { <br></p><p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify"> source(s_2514); <br></p><p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify"> filter(f_remote_linux1); <br></p><p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify"> destination(d_remote_linux1); <br></p><p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify"> };
</p>To use GeoIP in the above configuration, check this <a href="https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/geoip-parser.html">https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/geoip-parser.html</a><br><br></div>I am not expert on ELK, so from elasticsearch to kibana, you need to find out - how it should be. <br><div><div><p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify">
Regards</p><p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify">Dwijadas Dey<br></p><p class="gmail-western" style="margin-bottom:0.14in;line-height:115%" align="justify">
<br>
</p>
<br></div></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Apr 24, 2017 at 7:12 PM, C. L. Martinez <span dir="ltr"><<a href="mailto:carlopmart@gmail.com" target="_blank">carlopmart@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi all,<br>
<br>
I would like to drop Logstash collector from our ELK infrastructure and use syslog-ng instead. This ELK infrastructure collects, report and show dashboards about security devices: firewalls, anti-spam devices, etc.<br>
<br>
Most of these logs arrives from rsyslog collectors (deployed in several linux and BSD machines). I have seen in Balabit's blog page how this could be done: <a href="https://www.balabit.com/blog/how-to-parse-data-with-syslog-ng-store-in-elasticsearch-and-analyze-with-kibana/" rel="noreferrer" target="_blank">https://www.balabit.com/blog/<wbr>how-to-parse-data-with-syslog-<wbr>ng-store-in-elasticsearch-and-<wbr>analyze-with-kibana/</a> and <a href="https://www.balabit.com/blog/collecting-and-parsing-suricata-logs-using-syslog-ng/" rel="noreferrer" target="_blank">https://www.balabit.com/blog/<wbr>collecting-and-parsing-<wbr>suricata-logs-using-syslog-ng/</a><wbr>.<br>
<br>
The most important point here is to test all configured logstash filters inside syslog-ng: GeoIP patterns, some substitution params, etc. Any tips or tricks to accomplish this type of change?<br>
<br>
Many thanks.<br>
<span class="HOEnZb"><font color="#888888"><br>
--<br>
Greetings,<br>
C. L. Martinez<br>
______________________________<wbr>______________________________<wbr>__________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank">https://lists.balabit.hu/<wbr>mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank">http://www.balabit.com/<wbr>support/documentation/?<wbr>product=syslog-ng</a><br>
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank">http://www.balabit.com/wiki/<wbr>syslog-ng-faq</a><br>
<br>
</font></span></blockquote></div><br></div>