<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">Any feedback on summary would be appreciated. Thanks for all the help. </div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">1) Did not see this in any docs but, when compiling for systemd the systemd-devel package should be installed and either check for auto discovery or --enable-systemd.</div><div class=""><br class=""></div><div class="">2) Use elasticsearch2 for a destination. </div><blockquote style="margin: 0 0 0 40px; border: none; padding: 0px;" class=""><div class=""><div class="">destination d_es {</div></div><div class=""><div class=""> elasticsearch2(</div></div><div class=""><div class=""> index("syslog-ng_${YEAR}.${MONTH}.${DAY}")</div></div><div class=""><div class=""> type("syslog-ng") # Description: The type of the index. For example, type("test")</div></div><div class=""><div class=""> template("$(format-json --scope rfc5424 --exclude DATE --key ISODATE @timestamp=${ISODATE})")</div></div><div class=""><div class=""> port("9300")</div></div><div class=""><div class=""> server("localhost")</div></div><div class=""><div class=""> flush_limit("5000")</div></div><div class=""><div class=""> client_mode("node")</div></div><div class=""><div class=""> cluster(“syslog-ng")</div></div><div class=""><div class=""> custom_id("")</div></div><div class=""><div class=""> resource("/etc/elasticsearch/elasticsearch.yml”) # <b class="">YML Needs path.home like /usr/lib/systemd/system/elasticsearch.service</b></div></div><div class=""><div class=""> client_lib_dir("/usr/share/elasticsearch/lib")</div></div><div class=""><div class=""> concurrent_requests("1")</div></div><div class=""><div class=""> );</div></div><div class=""><div class="">};</div></div></blockquote><div class=""> </div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><div class="">[root@loghost etc]# cat /usr/lib/systemd/system/elasticsearch.service</div><div class="">[Unit]</div><div class="">Description=Elasticsearch</div><div class="">Documentation=<a href="http://www.elastic.co" class="">http://www.elastic.co</a></div><div class="">Wants=network-online.target</div><div class="">After=network-online.target</div><div class=""><br class=""></div><div class="">[Service]</div><div class="">Environment=ES_HOME=/usr/share/elasticsearch</div><div class="">Environment=CONF_DIR=/etc/elasticsearch</div><div class="">Environment=DATA_DIR=/var/lib/elasticsearch</div><div class="">Environment=LOG_DIR=/var/log/elasticsearch</div><div class="">Environment=PID_DIR=/var/run/elasticsearch</div><div class="">EnvironmentFile=-/etc/sysconfig/elasticsearch</div><div class=""><br class=""></div><div class="">WorkingDirectory=/usr/share/elasticsearch</div><div class=""><br class=""></div><div class="">User=elasticsearch</div><div class="">Group=elasticsearch</div><div class=""><br class=""></div><div class="">ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec</div><div class=""><br class=""></div><div class="">ExecStart=/usr/share/elasticsearch/bin/elasticsearch \</div><div class=""> -Des.pidfile=${PID_DIR}/elasticsearch.pid \</div><div class=""> -<b class="">Des.default.path.home</b>=${ES_HOME} \</div><div class=""> -Des.default.path.logs=${LOG_DIR} \</div><div class=""> -Des.default.path.data=${DATA_DIR} \</div><div class=""> -Des.default.path.conf=${CONF_DIR}</div><div class=""><br class=""></div><div class="">StandardOutput=journal</div><div class="">StandardError=inherit</div><div class=""><br class=""></div><div class=""># Specifies the maximum file descriptor number that can be opened by this process</div><div class="">LimitNOFILE=65535</div><div class=""><br class=""></div><div class=""># Specifies the maximum number of bytes of memory that may be locked into RAM</div><div class=""># Set to "infinity" if you use the 'bootstrap.mlockall: true' option</div><div class=""># in elasticsearch.yml and 'MAX_LOCKED_MEMORY=unlimited' in /etc/sysconfig/elasticsearch</div><div class="">#LimitMEMLOCK=infinity</div><div class=""><br class=""></div><div class=""># Disable timeout logic and wait until process is stopped</div><div class="">TimeoutStopSec=0</div><div class=""><br class=""></div><div class=""># SIGTERM signal is used to stop the Java process</div><div class="">KillSignal=SIGTERM</div><div class=""><br class=""></div><div class=""># Java process is never killed</div><div class="">SendSIGKILL=no</div><div class=""><br class=""></div><div class=""># When a JVM receives a SIGTERM signal it exits with code 143</div><div class="">SuccessExitStatus=143</div><div class=""><br class=""></div><div class="">[Install]</div><div class="">WantedBy=multi-user.target</div><div class=""><br class=""></div><div class=""># Built for Distribution: RPM-2.3.1 (rpm)</div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class="">tail -100f /var/log/messages</div><div class="">….</div><div class=""><div class="">Apr 13 11:23:03 loghost syslog-ng[10744]: syslog-ng starting up; version='3.8.0alpha0'</div><div class="">Apr 13 11:23:10 loghost syslog-ng[10744]: org.syslog_ng.elasticsearch_v2.client.ESClient.connect:61 - connecting to cluster, cluster_name='syslog-ng';</div><div class="">Apr 13 11:23:10 loghost syslog-ng[10744]: org.syslog_ng.elasticsearch_v2.client.ESClient.connect:71 - conneted to cluster, cluster_name='syslog-ng’;</div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><div class=""><b class="">NOTE</b>: I do not believe some options like server,port,.. are needed in node mode. </div><div class=""><span class="Apple-tab-span" style="white-space: pre;"> </span>What is the relationship to the ES service and syslog-ng in node mode? It looks like in node mode the es2 module write directly to the shard ? </div><div class=""> </div><div class=""><br class=""></div><div class=""><div class=""> 942 ? Ssl 2:23 /bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/elasticsearch-2.3.1.jar:/usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -Des.pidfile=/var/run/elasticsearch/elasticsearch.pid -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/var/lib/elasticsearch -Des.default.path.conf=/etc/elasticsearch</div><div class=""><br class=""></div><div class="">10744 ? Ssl 0:11 \_ /opt/syslog-ng/sbin/syslog-ng -f /opt/syslog-ng/etc/syslog-ng.conf -p /var/run/syslogd.pid</div></div><div class=""><br class=""></div></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><div class=""><br class=""></div><br class=""><div><blockquote type="cite" class=""><div class="">On Apr 13, 2016, at 10:49 AM, Scot Needy <<a href="mailto:scotrn@gmail.com" class="">scotrn@gmail.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class="">I assume you meant elasticsearch2 not elasticsearch_v2 ? elasticsearch2 seems to kinda work. <br class=""><br class="">Error parsing destination, destination plugin elasticsearch_v2 not found in /opt/syslog-ng/etc/static.destinations.conf at line 37, column 2:<br class=""> included from /opt/syslog-ng/etc/syslog-ng.conf line 63, column 35<br class=""><br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>elasticsearch_v2(<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>^^^^^^^^^^^^^^^^<br class=""><br class=""><br class=""><br class="">destination d_es {<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>elasticsearch2(<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span>index("syslog-ng_${YEAR}.${MONTH}.${DAY}")<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span>type("syslog-ng") # Description: The type of the index. For example, type("test")<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span>template("$(format-json --scope rfc5424 --exclude DATE --key ISODATE @timestamp=${ISODATE})")<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span>port("9300")<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span>server("localhost")<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span>flush_limit("5000")<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span>client_mode("node")<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span>cluster("")<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span>custom_id("")<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span>resource("")<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span>client_lib_dir("/usr/share/elasticsearch/lib")<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span><span class="Apple-tab-span" style="white-space:pre"> </span>concurrent_requests("1")<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>);<br class="">};<br class=""><br class="">/opt/syslog-ng/sbin/syslog-ng -f /opt/syslog-ng/etc/syslog-ng.conf -p /var/run/syslogd.pid -F -d --trace -v<br class="">….<br class="">[2016-04-13T10:19:22.529521] Log pattern database reloaded; file='/opt/syslog-ng/var/patterndb.xml', version='4', pub_date='2016-04-13'<br class="">[2016-04-13T10:19:22.651060] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/syslog-ng-core.jar;<br class="">[2016-04-13T10:19:22.651704] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/syslog-ng-core.jar;<br class="">[2016-04-13T10:19:22.651852] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/log4j-1.2.16.jar;<br class="">[2016-04-13T10:19:22.651972] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/syslog-ng-common.jar;<br class="">[2016-04-13T10:19:22.652125] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/dummy.jar;<br class="">[2016-04-13T10:19:22.652304] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/elastic-v2.jar;<br class="">[2016-04-13T10:19:22.652469] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/elastic.jar;<br class="">[2016-04-13T10:19:22.652606] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/hdfs.jar;<br class="">[2016-04-13T10:19:22.652736] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/http.jar;<br class="">[2016-04-13T10:19:22.652878] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/kafka.jar;<br class="">[2016-04-13T10:19:22.652998] Add path to classpath: /usr/share/elasticsearch/lib/HdrHistogram-2.1.6.jar;<br class="">[2016-04-13T10:19:22.653116] Add path to classpath: /usr/share/elasticsearch/lib/apache-log4j-extras-1.2.17.jar;<br class="">[2016-04-13T10:19:22.653253] Add path to classpath: /usr/share/elasticsearch/lib/commons-cli-1.3.1.jar;<br class="">[2016-04-13T10:19:22.653360] Add path to classpath: /usr/share/elasticsearch/lib/compiler-0.8.13.jar;<br class="">[2016-04-13T10:19:22.653490] Add path to classpath: /usr/share/elasticsearch/lib/compress-lzf-1.0.2.jar;<br class="">[2016-04-13T10:19:22.653601] Add path to classpath: /usr/share/elasticsearch/lib/elasticsearch-2.3.1.jar;<br class="">[2016-04-13T10:19:22.653750] Add path to classpath: /usr/share/elasticsearch/lib/guava-18.0.jar;<br class="">[2016-04-13T10:19:22.654190] Add path to classpath: /usr/share/elasticsearch/lib/hppc-0.7.1.jar;<br class="">[2016-04-13T10:19:22.654301] Add path to classpath: /usr/share/elasticsearch/lib/jackson-core-2.6.2.jar;<br class="">[2016-04-13T10:19:22.654446] Add path to classpath: /usr/share/elasticsearch/lib/jackson-dataformat-cbor-2.6.2.jar;<br class="">[2016-04-13T10:19:22.654576] Add path to classpath: /usr/share/elasticsearch/lib/jackson-dataformat-smile-2.6.2.jar;<br class="">[2016-04-13T10:19:22.654704] Add path to classpath: /usr/share/elasticsearch/lib/jackson-dataformat-yaml-2.6.2.jar;<br class="">[2016-04-13T10:19:22.654833] Add path to classpath: /usr/share/elasticsearch/lib/jna-4.1.0.jar;<br class="">[2016-04-13T10:19:22.654984] Add path to classpath: /usr/share/elasticsearch/lib/joda-convert-1.2.jar;<br class="">[2016-04-13T10:19:22.655108] Add path to classpath: /usr/share/elasticsearch/lib/joda-time-2.8.2.jar;<br class="">[2016-04-13T10:19:22.655222] Add path to classpath: /usr/share/elasticsearch/lib/jsr166e-1.1.0.jar;<br class="">[2016-04-13T10:19:22.655324] Add path to classpath: /usr/share/elasticsearch/lib/jts-1.13.jar;<br class="">[2016-04-13T10:19:22.655452] Add path to classpath: /usr/share/elasticsearch/lib/log4j-1.2.17.jar;<br class="">[2016-04-13T10:19:22.655557] Add path to classpath: /usr/share/elasticsearch/lib/lucene-analyzers-common-5.5.0.jar;<br class="">[2016-04-13T10:19:22.655684] Add path to classpath: /usr/share/elasticsearch/lib/lucene-backward-codecs-5.5.0.jar;<br class="">[2016-04-13T10:19:22.655977] Add path to classpath: /usr/share/elasticsearch/lib/lucene-core-5.5.0.jar;<br class="">[2016-04-13T10:19:22.656205] Add path to classpath: /usr/share/elasticsearch/lib/lucene-grouping-5.5.0.jar;<br class="">[2016-04-13T10:19:22.656322] Add path to classpath: /usr/share/elasticsearch/lib/lucene-highlighter-5.5.0.jar;<br class="">[2016-04-13T10:19:22.656462] Add path to classpath: /usr/share/elasticsearch/lib/lucene-join-5.5.0.jar;<br class="">[2016-04-13T10:19:22.656577] Add path to classpath: /usr/share/elasticsearch/lib/lucene-memory-5.5.0.jar;<br class="">[2016-04-13T10:19:22.656751] Add path to classpath: /usr/share/elasticsearch/lib/lucene-misc-5.5.0.jar;<br class="">[2016-04-13T10:19:22.656851] Add path to classpath: /usr/share/elasticsearch/lib/lucene-queries-5.5.0.jar;<br class="">[2016-04-13T10:19:22.656974] Add path to classpath: /usr/share/elasticsearch/lib/lucene-queryparser-5.5.0.jar;<br class="">[2016-04-13T10:19:22.657093] Add path to classpath: /usr/share/elasticsearch/lib/lucene-sandbox-5.5.0.jar;<br class="">[2016-04-13T10:19:22.657209] Add path to classpath: /usr/share/elasticsearch/lib/lucene-spatial-5.5.0.jar;<br class="">[2016-04-13T10:19:22.657318] Add path to classpath: /usr/share/elasticsearch/lib/lucene-spatial3d-5.5.0.jar;<br class="">[2016-04-13T10:19:22.657448] Add path to classpath: /usr/share/elasticsearch/lib/lucene-suggest-5.5.0.jar;<br class="">[2016-04-13T10:19:22.657616] Add path to classpath: /usr/share/elasticsearch/lib/netty-3.10.5.Final.jar;<br class="">[2016-04-13T10:19:22.657743] Add path to classpath: /usr/share/elasticsearch/lib/securesm-1.0.jar;<br class="">[2016-04-13T10:19:22.657855] Add path to classpath: /usr/share/elasticsearch/lib/snakeyaml-1.15.jar;<br class="">[2016-04-13T10:19:22.657979] Add path to classpath: /usr/share/elasticsearch/lib/spatial4j-0.5.jar;<br class="">[2016-04-13T10:19:22.658075] Add path to classpath: /usr/share/elasticsearch/lib/t-digest-3.0.jar;<br class="">[2016-04-13T10:19:22.731812] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/syslog-ng-core.jar;<br class="">[2016-04-13T10:19:22.900320] Exception occured: java.lang.IllegalStateException: path.home is not configured<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>at org.elasticsearch.env.Environment.<init>(Environment.java:101)<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:81)<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>at org.elasticsearch.node.Node.<init>(Node.java:140)<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:143)<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>at org.elasticsearch.node.NodeBuilder.node(NodeBuilder.java:150)<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>at org.syslog_ng.elasticsearch_v2.client.ESNodeClient.createClient(ESNodeClient.java:78)<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>at org.syslog_ng.elasticsearch_v2.client.ESClient.init(ESClient.java:100)<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>at org.syslog_ng.elasticsearch_v2.ElasticSearchDestination.init(ElasticSearchDestination.java:63)<br class=""><span class="Apple-tab-span" style="white-space:pre"> </span>at org.syslog_ng.LogPipe.initProxy(LogPipe.java:64)<br class=""><br class=""><br class=""><blockquote type="cite" class="">On Apr 13, 2016, at 8:53 AM, Fabien Wernli <<a href="mailto:wernli@in2p3.fr" class="">wernli@in2p3.fr</a>> wrote:<br class=""><br class="">On Wed, Apr 13, 2016 at 08:48:47AM -0400, Scot wrote:<br class=""><blockquote type="cite" class="">How do I specify ?<br class=""></blockquote><br class="">there's two destinations: elasticsearch and elasticsearch_v2<br class=""><br class="">______________________________________________________________________________<br class="">Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" class="">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br class="">Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" class="">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br class="">FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" class="">http://www.balabit.com/wiki/syslog-ng-faq</a><br class=""><br class=""></blockquote><br class=""></div></blockquote></div><br class=""></body></html>