Hi Martinez You need to install syslog-ng first. Then create configuration file for your linux and BSD machines inside /etc/syslog-ng/conf.d Say you have configured one linux machine that forwards logs to this centralized syslog-ng server then you can use following configuration so that logs gets transferred to kibana. Forwarder (Linux machine) -> Syslog-ng ( Centralize log collector) -> Elastic search -> Kibana One such example can be like. # vi /etc/syslog-ng/conf.d/remote-linux-1.conf source s_2514 { tcp(port(2514)); }; # Assuming remote linux machine forwarding logs to syslog-ng server's tcp port 2514. destination d_remote_linux1 { tcp("127.0.0.1" port(9200) template("$(format-json --scope selected_macros --scope nv_pairs)\n")); }; # Change the above IP -> 127.0.0.1 to your elastic server's IP filter f_remote_linux1 { '' your filter " ; }; # Apply whatever filter you want, you can use multiple filters as well. log { source(s_2514); filter(f_remote_linux1); destination(d_remote_linux1); }; To use GeoIP in the above configuration, check this https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-o... I am not expert on ELK, so from elasticsearch to kibana, you need to find out - how it should be. Regards Dwijadas Dey On Mon, Apr 24, 2017 at 7:12 PM, C. L. Martinez <carlopmart@gmail.com> wrote:
Hi all,
I would like to drop Logstash collector from our ELK infrastructure and use syslog-ng instead. This ELK infrastructure collects, report and show dashboards about security devices: firewalls, anti-spam devices, etc.
Most of these logs arrives from rsyslog collectors (deployed in several linux and BSD machines). I have seen in Balabit's blog page how this could be done: https://www.balabit.com/blog/how-to-parse-data-with-syslog- ng-store-in-elasticsearch-and-analyze-with-kibana/ and https://www.balabit.com/blog/collecting-and-parsing- suricata-logs-using-syslog-ng/.
The most important point here is to test all configured logstash filters inside syslog-ng: GeoIP patterns, some substitution params, etc. Any tips or tricks to accomplish this type of change?
Many thanks.
-- Greetings, C. L. Martinez ____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq