[syslog-ng] Convert logstash filters to syslog-ng

[Dwijadas Dey]

Hi
    Martinez
                  You need to install syslog-ng first. Then create
configuration file for your linux and BSD machines inside
/etc/syslog-ng/conf.d

Say you have configured one linux machine that forwards logs to this
centralized syslog-ng server then you can use following configuration so
that logs gets transferred to kibana.

Forwarder (Linux machine) -> Syslog-ng ( Centralize log collector) ->
Elastic search -> Kibana

One such example can be like.

# vi /etc/syslog-ng/conf.d/remote-linux-1.conf

source s_2514 { tcp(port(2514)); };

# Assuming remote linux machine forwarding logs to syslog-ng server's tcp
port 2514.

destination d_remote_linux1 { tcp("127.0.0.1" port(9200)
template("$(format-json --scope selected_macros --scope nv_pairs)\n")); };

# Change the above IP -> 127.0.0.1 to your elastic server's IP

filter f_remote_linux1 {  '' your filter " ; };

# Apply whatever filter you want, you can use multiple filters as well.

log {

         source(s_2514);

         filter(f_remote_linux1);

         destination(d_remote_linux1);

    };
To use GeoIP in the above configuration, check this
https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/geoip-parser.html

I am not expert on ELK, so from elasticsearch to kibana, you need to find
out - how it should be.

Regards

Dwijadas Dey




On Mon, Apr 24, 2017 at 7:12 PM, C. L. Martinez <carlopmart at gmail.com>
wrote:

> Hi all,
>
>  I would like to drop Logstash collector from our ELK infrastructure and
> use syslog-ng instead. This ELK infrastructure collects, report and show
> dashboards about security devices: firewalls, anti-spam devices, etc.
>
>  Most of these logs arrives from rsyslog collectors (deployed in several
> linux and BSD machines). I have seen in Balabit's blog page how this could
> be done: https://www.balabit.com/blog/how-to-parse-data-with-syslog-
> ng-store-in-elasticsearch-and-analyze-with-kibana/ and
> https://www.balabit.com/blog/collecting-and-parsing-
> suricata-logs-using-syslog-ng/.
>
>  The most important point here is to test all configured logstash filters
> inside syslog-ng: GeoIP patterns, some substitution params, etc. Any tips
> or tricks to accomplish this type of change?
>
> Many thanks.
>
> --
> Greetings,
> C. L. Martinez
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20170424/a64fe36c/attachment.html>