[syslog-ng] How to export syslogng to elasticsearch

[Mik J]

Hello,

I've read many documentation about how to export syslogng to elasticsearch but still don't get it

# syslog-ng -V
syslog-ng 3.7.2
Installer-Version: 3.7.2
Revision:
Compile-Date: May  8 2017 10:54:55
Available-Modules: afprog,afsocket,afsql,afuser,basicfuncs,confgen,csvparser,dbparser,graphite,json-plugin,kvformat,linux-kmsg-format,pseudofile,syslogformat,system-source,affile,cryptofuncs
Enable-Debug: off
Enable-GProf: off
Enable-Memtrace: off
Enable-IPv6: on
Enable-Spoof-Source: off
Enable-TCP-Wrapper: off
Enable-Linux-Caps: off


In syslog-ng.conf I need to add:

@module mod-java
@include "scl.conf"

In scl.conf I have (I kept the default configuration)

@define scl-root "`syslog-ng-data`/include/scl"
@define include-path "`include-path`:`syslog-ng-data`/include"
@include 'scl/*/*.conf'
I don't understand really what should I configure.

I also downloaded gradle
# ls /usr/local/gradle-3.5

And Java
# /usr/local/jdk-1.8.0/bin/java -version
openjdk version "1.8.0_121"
OpenJDK Runtime Environment (build 1.8.0_121-b13)
OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode)



Back to syslog-ng.conf I added
# Test Elasticsearch
filter f_MyTest { host("1.1.1.1"); };
log {
source(s_net);
filter(f_MyTest);
destination(d_MyTest);
};

destination d_elastic {
 elasticsearch(
 index("syslog-ng_${YEAR}.${MONTH}.${DAY}")
 type("test")
 cluster("syslog-ng")
 client_mode("transport")
 custom_id("${UNIQID}")
 flush-limit("10000")
 );
};
log {
 source(s_net);
 filter(f_MyTest);
 destination(d_elastic);
 flags(flow-control);
};


But starting the daemon gives me this error
# syslog-ng
[2017-05-18T16:43:42.465496] Plugin module not found in 'module-path'; module-path='/usr/local/lib/syslog-ng', module='mod-java'
Error parsing destination, destination plugin elasticsearch not found in /etc/syslog-ng/syslog-ng.conf at line 171, column 2:
elasticsearch(
^^^^^^^^^^^^^