<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class="">Hi All,<div class="">That worked for me too but I have few questions:</div><div class=""><br class=""></div><div class="">- Is this the expected behaviour?</div><div class="">- Do we still need to add the *.jar library files from the ES distribution?</div><div class="">- The client-lib-dir function seems to need *.jar when multiple paths are specified, apart from the last path in the line - is this correct?</div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">My path in the ES destination:</div><div class=""><br class=""></div><div class="">client-lib-dir(“/esjarfiles/*.jar:/usr/lib/syslog-ng/3.13/java-modules/elastic-jest-client/*.jar:/usr/lib/syslog-ng/3.13/java-modules/“)</div><div class=""><br class=""></div><div class="">Thanks,</div><div class="">Marco</div><div class=""><br class=""><div><blockquote type="cite" class=""><div class="">On 14 Dec 2017, at 23:08, hari ram <<a href="mailto:hariram@hotmail.com" class="">hariram@hotmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><div class="">root@ES6:/etc/syslog-ng# more syslog-ng.conf</div><div class="">@version:3.13</div><div class="">@module mod-java</div><div class="">@include "scl.conf"</div><div class="">options {</div><div class=""> flush_lines(0);</div><div class=""> keep_hostname(yes);</div><div class=""> normalize_hostnames(yes);</div><div class=""> threaded(yes);</div><div class="">};</div><div class="">source s_local { system(); internal(); };</div><div class="">source s_network { syslog(transport(tcp)); };</div><div class="">destination d_all { file ("/var/log/all.log"); };</div><div class="">destination d_elastic {</div><div class=""> elasticsearch2(</div><div class=""> client-lib-dir("/usr/lib/syslog-ng/3.13/java-modules/elastic-jest-client/*.jar:/usr/share/elasticsearch/lib/:/usr/lib/syslog-ng/3.13/java-modules/") --- adding path </div><div class=""> client_mode("http")</div><div class=""> cluster_url("<a href="http://192.168.1.75:9200/" class="">http://192.168.1.75:9200</a>")</div><div class=""> index("syslog-ng_${YEAR}.${MONTH}.${DAY}")</div><div class=""> type("syslog")</div><div class=""> cluster("test")</div><div class=""> flush-limit("1000")</div><div class=""> template("$(format-json --scope rfc5424 --scope nv-pairs --exclude DATE --key ISODATE)")</div><div class=""> time-zone("UTC")</div><div class=""> );</div><div class="">};</div><div class="">log { source(s_network); destination(d_elastic); };</div><div class="">log { source(s_local); destination(d_all); };</div><div class=""><br class=""></div><br class=""></div><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><br class=""></div><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class="">fix the error, but i will test and come back.</div><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><br class=""></div><div style="font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class="">R!</div><hr tabindex="-1" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; display: inline-block; width: 756.546875px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class=""></span><div id="divRplyFwdMsg" dir="ltr" style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><font face="Calibri, sans-serif" style="font-size: 11pt;" class=""><b class="">From:</b><span class="Apple-converted-space"> </span>syslog-ng <<a href="mailto:syslog-ng-bounces@lists.balabit.hu" class="">syslog-ng-bounces@lists.balabit.hu</a>> on behalf of hari ram <<a href="mailto:hariram@hotmail.com" class="">hariram@hotmail.com</a>><br class=""><b class="">Sent:</b><span class="Apple-converted-space"> </span>14 December 2017 23:04<br class=""><b class="">To:</b><span class="Apple-converted-space"> </span><a href="mailto:syslog-ng@lists.balabit.hu" class="">syslog-ng@lists.balabit.hu</a><br class=""><b class="">Subject:</b><span class="Apple-converted-space"> </span>[syslog-ng] SYSLOG-NG issue with ES 6.X</font><div class=""> </div></div><div style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class="">Hi</div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><br class=""></div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class="">I have installed SYSLOG-NG 3.13.2 on ubunutu, try to send logs to ES 6.0 i failed to do so, here is my inputs.</div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><br class=""></div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><div class="">root@ES6:/etc/syslog-ng# syslog-ng -V</div><div class="">syslog-ng 3 (3.13.2)</div><div class="">Config version: 3.13</div><div class="">Installer-Version: 3.13.2</div><div class="">Revision: 3.13.2-1</div><div class="">Compile-Date: Dec 5 2017 13:24:07</div><div class="">Module-Directory: /usr/lib/syslog-ng/3.13</div><div class="">Module-Path: /usr/lib/syslog-ng/3.13</div><div class="">Available-Modules: afuser,mod-python,afstomp,http,afsql,disk-buffer,mod-java,cef,pseudofile,sdjournal,kvformat,xml,csvparser,snmptrapd-parser,appmodel,confgen,pacctformat,linux-kmsg-format,dbparser,system-source,map-value-pairs,add-contextual-data,date,syslogformat,afamqp,geoip2-plugin,tfgetent,graphite,afmongodb,cryptofuncs,geoip-plugin,afsmtp,afsocket,redis,affile,stardate,basicfuncs,riemann,json-plugin,tags-parser,afprog</div><div class="">Enable-Debug: off</div><div class="">Enable-GProf: off</div><div class="">Enable-Memtrace: off</div><div class="">Enable-IPv6: on</div><div class="">Enable-Spoof-Source: on</div><div class="">Enable-TCP-Wrapper: on</div><div class="">Enable-Linux-Caps: on</div><div class="">Enable-Systemd: on</div><div class=""><br class=""></div>===</div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><br class=""></div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><div class="">root@ES6:/etc/syslog-ng# more syslog-ng.conf</div><div class="">@version:3.13</div><div class="">@module mod-java</div><div class="">@include "scl.conf"</div><div class="">options {</div><div class=""> flush_lines(0);</div><div class=""> keep_hostname(yes);</div><div class=""> normalize_hostnames(yes);</div><div class=""> threaded(yes);</div><div class="">};</div><div class="">source s_local { system(); internal(); };</div><div class="">source s_network { syslog(transport(tcp)); };</div><div class="">destination d_all { file ("/var/log/all.log"); };</div><div class="">destination d_elastic {</div><div class=""> elasticsearch2(</div><div class=""> client-lib-dir("/usr/share/elasticsearch/lib/:/usr/lib/syslog-ng/3.13/java-modules/")</div><div class=""> client_mode("http")</div><div class=""> cluster_url("<a href="http://192.168.1.75:9200" class="">http://192.168.1.75:9200</a>")</div><div class=""> index("syslog-ng_${YEAR}.${MONTH}.${DAY}")</div><div class=""> type("syslog")</div><div class=""> cluster("test")</div><div class=""> flush-limit("1000")</div><div class=""> template("$(format-json --scope rfc5424 --scope nv-pairs --exclude DATE --key ISODATE)")</div><div class=""> time-zone("UTC")</div><div class=""> );</div><div class="">};</div><div class="">log { source(s_network); destination(d_elastic); };</div><div class="">log { source(s_local); destination(d_all); };</div><div class=""><br class=""></div><br class=""></div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class="">===</div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><br class=""></div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><div class="">root@ES6:/etc/syslog-ng# ls /usr/share/elasticsearch/lib/</div><div class="">elasticsearch-6.0.1.jar jackson-dataformat-smile-2.8.6.jar jopt-simple-5.0.2.jar lucene-analyzers-common-7.0.1.jar lucene-join-7.0.1.jar lucene-sandbox-7.0.1.jar plugin-cli-6.0.1.jar</div><div class="">HdrHistogram-2.1.9.jar jackson-dataformat-yaml-2.8.6.jar jts-1.13.jar lucene-backward-codecs-7.0.1.jar lucene-memory-7.0.1.jar lucene-spatial3d-7.0.1.jar securesm-1.2.jar</div><div class="">hppc-0.7.1.jar java-version-checker-6.0.1.jar log4j-1.2-api-2.9.1.jar lucene-core-7.0.1.jar lucene-misc-7.0.1.jar lucene-spatial-7.0.1.jar snakeyaml-1.15.jar</div><div class="">jackson-core-2.8.6.jar jna-4.4.0-1.jar log4j-api-2.9.1.jar lucene-grouping-7.0.1.jar lucene-queries-7.0.1.jar lucene-spatial-extras-7.0.1.jar spatial4j-0.6.jar</div><div class="">jackson-dataformat-cbor-2.8.6.jar joda-time-2.9.5.jar log4j-core-2.9.1.jar lucene-highlighter-7.0.1.jar lucene-queryparser-7.0.1.jar lucene-suggest-7.0.1.jar t-digest-3.0.jar</div><div class=""><br class=""></div><br class=""></div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class="">====</div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><br class=""></div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><div class="">root@ES6:/etc/syslog-ng# ls /usr/lib/syslog-ng/3.13/java-modules/</div><div class="">elastic.jar elastic-jest-client elastic-v2.jar hdfs.jar http.jar kafka.jar log4j-1.2.16.jar syslog-ng-common.jar syslog-ng-core.jar</div><div class=""><br class=""></div></div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class="">==</div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><br class=""></div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class="">root@ES6:/etc/syslog-ng# syslog-ng -Fevd<br class=""></div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><br class=""></div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><div class="">[2017-12-14T23:04:21.552408] Compiling #unnamed sequence [log] at [source generator system:14:12]</div><div class="">[2017-12-14T23:04:21.552510] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:10:35]</div><div class="">[2017-12-14T23:04:21.552632] Compiling d_all reference [destination] at [/etc/syslog-ng/syslog-ng.conf:27:24]</div><div class="">[2017-12-14T23:04:21.552715] Compiling d_all sequence [destination] at [/etc/syslog-ng/syslog-ng.conf:12:1]</div><div class="">[2017-12-14T23:04:21.552781] Compiling #unnamed junction [log] at [/etc/syslog-ng/syslog-ng.conf:12:20]</div><div class="">[2017-12-14T23:04:21.552884] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:12:21]</div><div class="">[2017-12-14T23:04:21.553211] Module loaded and initialized successfully; module='syslogformat'</div><div class="">[2017-12-14T23:04:21.553425] Processing the time zone file (32bit part); filename='/usr/share/zoneinfo/UTC'</div><div class="">[2017-12-14T23:04:21.671696] Add path to classpath: /usr/lib/syslog-ng/3.13/java-modules/syslog-ng-core.jar;</div><div class="">[2017-12-14T23:04:21.672418] Add path to classpath: /usr/share/elasticsearch/lib/;</div><div class="">[2017-12-14T23:04:21.673641] Add path to classpath: /usr/lib/syslog-ng/3.13/java-modules/syslog-ng-core.jar;</div><div class="">[2017-12-14T23:04:21.673912] Add path to classpath: /usr/lib/syslog-ng/3.13/java-modules/syslog-ng-common.jar;</div><div class="">[2017-12-14T23:04:21.674218] Add path to classpath: /usr/lib/syslog-ng/3.13/java-modules/hdfs.jar;</div><div class="">[2017-12-14T23:04:21.674704] Add path to classpath: /usr/lib/syslog-ng/3.13/java-modules/http.jar;</div><div class="">[2017-12-14T23:04:21.675858] Add path to classpath: /usr/lib/syslog-ng/3.13/java-modules/kafka.jar;</div><div class="">[2017-12-14T23:04:21.676116] Add path to classpath: /usr/lib/syslog-ng/3.13/java-modules/log4j-1.2.16.jar;</div><div class="">[2017-12-14T23:04:21.676322] Add path to classpath: /usr/lib/syslog-ng/3.13/java-modules/elastic-v2.jar;</div><div class="">[2017-12-14T23:04:21.676484] Add path to classpath: /usr/lib/syslog-ng/3.13/java-modules/elastic.jar;</div><div class="">[2017-12-14T23:04:21.741649] Add path to classpath: /usr/lib/syslog-ng/3.13/java-modules/syslog-ng-core.jar;</div><div class="">[2017-12-14T23:04:21.746168] Error initializing message pipeline; plugin name='java', location='#buffer:2:3'</div><br class=""></div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><br class=""></div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class="">Any suggestions ?</div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><br class=""></div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class="">R!</div><div style="font-family: Calibri, Helvetica, sans-serif; font-size: 12pt; background-color: rgba(0, 0, 0, 0);" class=""><br class=""></div></div><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">______________________________________________________________________________</span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" class="">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a></span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" class="">http://www.balabit.com/support/documentation/?product=syslog-ng</a></span><br style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px;" class=""><span style="font-family: Helvetica; font-size: 12px; font-style: normal; font-variant-caps: normal; font-weight: normal; letter-spacing: normal; text-align: start; text-indent: 0px; text-transform: none; white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; float: none; display: inline !important;" class="">FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" class="">http://www.balabit.com/wiki/syslog-ng-faq</a></span></div></blockquote></div><br class=""></div></body></html>