Hi
    Martinez
                  You need to install syslog-ng first. Then create configuration file for your linux and BSD machines inside /etc/syslog-ng/conf.d

Say you have configured one linux machine that forwards logs to this centralized syslog-ng server then you can use following configuration so that logs gets transferred to kibana.

Forwarder (Linux machine) -> Syslog-ng ( Centralize log collector) -> Elastic search -> Kibana

One such example can be like.

# vi /etc/syslog-ng/conf.d/remote-linux-1.conf

source s_2514 { tcp(port(2514)); };

# Assuming remote linux machine forwarding logs to syslog-ng server's tcp port 2514.

destination d_remote_linux1 { tcp("127.0.0.1" port(9200) template("$(format-json --scope selected_macros --scope nv_pairs)\n")); };

# Change the above IP -> 127.0.0.1 to your elastic server's IP

filter f_remote_linux1 {  '' your filter " ; };

# Apply whatever filter you want, you can use multiple filters as well.

log {

         source(s_2514);

         filter(f_remote_linux1);

         destination(d_remote_linux1);

    };

To use GeoIP in the above configuration, check this https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/geoip-parser.html

I am not expert on ELK, so from elasticsearch to kibana, you need to find out - how it should be.

Regards

Dwijadas Dey




On Mon, Apr 24, 2017 at 7:12 PM, C. L. Martinez <carlopmart@gmail.com> wrote:
Hi all,

 I would like to drop Logstash collector from our ELK infrastructure and use syslog-ng instead. This ELK infrastructure collects, report and show dashboards about security devices: firewalls, anti-spam devices, etc.

 Most of these logs arrives from rsyslog collectors (deployed in several linux and BSD machines). I have seen in Balabit's blog page how this could be done: https://www.balabit.com/blog/how-to-parse-data-with-syslog-ng-store-in-elasticsearch-and-analyze-with-kibana/ and https://www.balabit.com/blog/collecting-and-parsing-suricata-logs-using-syslog-ng/.

 The most important point here is to test all configured logstash filters inside syslog-ng: GeoIP patterns, some substitution params, etc. Any tips or tricks to accomplish this type of change?

Many thanks.

--
Greetings,
C. L. Martinez
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq