[syslog-ng] systemd syslog-ng does not start CentOS 7.2.1511 [ SOLVED ]
Scot Needy
scotrn at gmail.com
Wed Apr 13 17:32:47 CEST 2016
Any feedback on summary would be appreciated. Thanks for all the help.
1) Did not see this in any docs but, when compiling for systemd the systemd-devel package should be installed and either check for auto discovery or --enable-systemd.
2) Use elasticsearch2 for a destination.
destination d_es {
elasticsearch2(
index("syslog-ng_${YEAR}.${MONTH}.${DAY}")
type("syslog-ng") # Description: The type of the index. For example, type("test")
template("$(format-json --scope rfc5424 --exclude DATE --key ISODATE @timestamp=${ISODATE})")
port("9300")
server("localhost")
flush_limit("5000")
client_mode("node")
cluster(“syslog-ng")
custom_id("")
resource("/etc/elasticsearch/elasticsearch.yml”) # YML Needs path.home like /usr/lib/systemd/system/elasticsearch.service
client_lib_dir("/usr/share/elasticsearch/lib")
concurrent_requests("1")
);
};
[root at loghost etc]# cat /usr/lib/systemd/system/elasticsearch.service
[Unit]
Description=Elasticsearch
Documentation=http://www.elastic.co
Wants=network-online.target
After=network-online.target
[Service]
Environment=ES_HOME=/usr/share/elasticsearch
Environment=CONF_DIR=/etc/elasticsearch
Environment=DATA_DIR=/var/lib/elasticsearch
Environment=LOG_DIR=/var/log/elasticsearch
Environment=PID_DIR=/var/run/elasticsearch
EnvironmentFile=-/etc/sysconfig/elasticsearch
WorkingDirectory=/usr/share/elasticsearch
User=elasticsearch
Group=elasticsearch
ExecStartPre=/usr/share/elasticsearch/bin/elasticsearch-systemd-pre-exec
ExecStart=/usr/share/elasticsearch/bin/elasticsearch \
-Des.pidfile=${PID_DIR}/elasticsearch.pid \
-Des.default.path.home=${ES_HOME} \
-Des.default.path.logs=${LOG_DIR} \
-Des.default.path.data=${DATA_DIR} \
-Des.default.path.conf=${CONF_DIR}
StandardOutput=journal
StandardError=inherit
# Specifies the maximum file descriptor number that can be opened by this process
LimitNOFILE=65535
# Specifies the maximum number of bytes of memory that may be locked into RAM
# Set to "infinity" if you use the 'bootstrap.mlockall: true' option
# in elasticsearch.yml and 'MAX_LOCKED_MEMORY=unlimited' in /etc/sysconfig/elasticsearch
#LimitMEMLOCK=infinity
# Disable timeout logic and wait until process is stopped
TimeoutStopSec=0
# SIGTERM signal is used to stop the Java process
KillSignal=SIGTERM
# Java process is never killed
SendSIGKILL=no
# When a JVM receives a SIGTERM signal it exits with code 143
SuccessExitStatus=143
[Install]
WantedBy=multi-user.target
# Built for Distribution: RPM-2.3.1 (rpm)
tail -100f /var/log/messages
….
Apr 13 11:23:03 loghost syslog-ng[10744]: syslog-ng starting up; version='3.8.0alpha0'
Apr 13 11:23:10 loghost syslog-ng[10744]: org.syslog_ng.elasticsearch_v2.client.ESClient.connect:61 - connecting to cluster, cluster_name='syslog-ng';
Apr 13 11:23:10 loghost syslog-ng[10744]: org.syslog_ng.elasticsearch_v2.client.ESClient.connect:71 - conneted to cluster, cluster_name='syslog-ng’;
NOTE: I do not believe some options like server,port,.. are needed in node mode.
What is the relationship to the ES service and syslog-ng in node mode? It looks like in node mode the es2 module write directly to the shard ?
942 ? Ssl 2:23 /bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/elasticsearch-2.3.1.jar:/usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -Des.pidfile=/var/run/elasticsearch/elasticsearch.pid -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/var/lib/elasticsearch -Des.default.path.conf=/etc/elasticsearch
10744 ? Ssl 0:11 \_ /opt/syslog-ng/sbin/syslog-ng -f /opt/syslog-ng/etc/syslog-ng.conf -p /var/run/syslogd.pid
> On Apr 13, 2016, at 10:49 AM, Scot Needy <scotrn at gmail.com> wrote:
>
> I assume you meant elasticsearch2 not elasticsearch_v2 ? elasticsearch2 seems to kinda work.
>
> Error parsing destination, destination plugin elasticsearch_v2 not found in /opt/syslog-ng/etc/static.destinations.conf at line 37, column 2:
> included from /opt/syslog-ng/etc/syslog-ng.conf line 63, column 35
>
> elasticsearch_v2(
> ^^^^^^^^^^^^^^^^
>
>
>
> destination d_es {
> elasticsearch2(
> index("syslog-ng_${YEAR}.${MONTH}.${DAY}")
> type("syslog-ng") # Description: The type of the index. For example, type("test")
> template("$(format-json --scope rfc5424 --exclude DATE --key ISODATE @timestamp=${ISODATE})")
> port("9300")
> server("localhost")
> flush_limit("5000")
> client_mode("node")
> cluster("")
> custom_id("")
> resource("")
> client_lib_dir("/usr/share/elasticsearch/lib")
> concurrent_requests("1")
> );
> };
>
> /opt/syslog-ng/sbin/syslog-ng -f /opt/syslog-ng/etc/syslog-ng.conf -p /var/run/syslogd.pid -F -d --trace -v
> ….
> [2016-04-13T10:19:22.529521] Log pattern database reloaded; file='/opt/syslog-ng/var/patterndb.xml', version='4', pub_date='2016-04-13'
> [2016-04-13T10:19:22.651060] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/syslog-ng-core.jar;
> [2016-04-13T10:19:22.651704] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/syslog-ng-core.jar;
> [2016-04-13T10:19:22.651852] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/log4j-1.2.16.jar;
> [2016-04-13T10:19:22.651972] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/syslog-ng-common.jar;
> [2016-04-13T10:19:22.652125] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/dummy.jar;
> [2016-04-13T10:19:22.652304] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/elastic-v2.jar;
> [2016-04-13T10:19:22.652469] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/elastic.jar;
> [2016-04-13T10:19:22.652606] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/hdfs.jar;
> [2016-04-13T10:19:22.652736] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/http.jar;
> [2016-04-13T10:19:22.652878] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/kafka.jar;
> [2016-04-13T10:19:22.652998] Add path to classpath: /usr/share/elasticsearch/lib/HdrHistogram-2.1.6.jar;
> [2016-04-13T10:19:22.653116] Add path to classpath: /usr/share/elasticsearch/lib/apache-log4j-extras-1.2.17.jar;
> [2016-04-13T10:19:22.653253] Add path to classpath: /usr/share/elasticsearch/lib/commons-cli-1.3.1.jar;
> [2016-04-13T10:19:22.653360] Add path to classpath: /usr/share/elasticsearch/lib/compiler-0.8.13.jar;
> [2016-04-13T10:19:22.653490] Add path to classpath: /usr/share/elasticsearch/lib/compress-lzf-1.0.2.jar;
> [2016-04-13T10:19:22.653601] Add path to classpath: /usr/share/elasticsearch/lib/elasticsearch-2.3.1.jar;
> [2016-04-13T10:19:22.653750] Add path to classpath: /usr/share/elasticsearch/lib/guava-18.0.jar;
> [2016-04-13T10:19:22.654190] Add path to classpath: /usr/share/elasticsearch/lib/hppc-0.7.1.jar;
> [2016-04-13T10:19:22.654301] Add path to classpath: /usr/share/elasticsearch/lib/jackson-core-2.6.2.jar;
> [2016-04-13T10:19:22.654446] Add path to classpath: /usr/share/elasticsearch/lib/jackson-dataformat-cbor-2.6.2.jar;
> [2016-04-13T10:19:22.654576] Add path to classpath: /usr/share/elasticsearch/lib/jackson-dataformat-smile-2.6.2.jar;
> [2016-04-13T10:19:22.654704] Add path to classpath: /usr/share/elasticsearch/lib/jackson-dataformat-yaml-2.6.2.jar;
> [2016-04-13T10:19:22.654833] Add path to classpath: /usr/share/elasticsearch/lib/jna-4.1.0.jar;
> [2016-04-13T10:19:22.654984] Add path to classpath: /usr/share/elasticsearch/lib/joda-convert-1.2.jar;
> [2016-04-13T10:19:22.655108] Add path to classpath: /usr/share/elasticsearch/lib/joda-time-2.8.2.jar;
> [2016-04-13T10:19:22.655222] Add path to classpath: /usr/share/elasticsearch/lib/jsr166e-1.1.0.jar;
> [2016-04-13T10:19:22.655324] Add path to classpath: /usr/share/elasticsearch/lib/jts-1.13.jar;
> [2016-04-13T10:19:22.655452] Add path to classpath: /usr/share/elasticsearch/lib/log4j-1.2.17.jar;
> [2016-04-13T10:19:22.655557] Add path to classpath: /usr/share/elasticsearch/lib/lucene-analyzers-common-5.5.0.jar;
> [2016-04-13T10:19:22.655684] Add path to classpath: /usr/share/elasticsearch/lib/lucene-backward-codecs-5.5.0.jar;
> [2016-04-13T10:19:22.655977] Add path to classpath: /usr/share/elasticsearch/lib/lucene-core-5.5.0.jar;
> [2016-04-13T10:19:22.656205] Add path to classpath: /usr/share/elasticsearch/lib/lucene-grouping-5.5.0.jar;
> [2016-04-13T10:19:22.656322] Add path to classpath: /usr/share/elasticsearch/lib/lucene-highlighter-5.5.0.jar;
> [2016-04-13T10:19:22.656462] Add path to classpath: /usr/share/elasticsearch/lib/lucene-join-5.5.0.jar;
> [2016-04-13T10:19:22.656577] Add path to classpath: /usr/share/elasticsearch/lib/lucene-memory-5.5.0.jar;
> [2016-04-13T10:19:22.656751] Add path to classpath: /usr/share/elasticsearch/lib/lucene-misc-5.5.0.jar;
> [2016-04-13T10:19:22.656851] Add path to classpath: /usr/share/elasticsearch/lib/lucene-queries-5.5.0.jar;
> [2016-04-13T10:19:22.656974] Add path to classpath: /usr/share/elasticsearch/lib/lucene-queryparser-5.5.0.jar;
> [2016-04-13T10:19:22.657093] Add path to classpath: /usr/share/elasticsearch/lib/lucene-sandbox-5.5.0.jar;
> [2016-04-13T10:19:22.657209] Add path to classpath: /usr/share/elasticsearch/lib/lucene-spatial-5.5.0.jar;
> [2016-04-13T10:19:22.657318] Add path to classpath: /usr/share/elasticsearch/lib/lucene-spatial3d-5.5.0.jar;
> [2016-04-13T10:19:22.657448] Add path to classpath: /usr/share/elasticsearch/lib/lucene-suggest-5.5.0.jar;
> [2016-04-13T10:19:22.657616] Add path to classpath: /usr/share/elasticsearch/lib/netty-3.10.5.Final.jar;
> [2016-04-13T10:19:22.657743] Add path to classpath: /usr/share/elasticsearch/lib/securesm-1.0.jar;
> [2016-04-13T10:19:22.657855] Add path to classpath: /usr/share/elasticsearch/lib/snakeyaml-1.15.jar;
> [2016-04-13T10:19:22.657979] Add path to classpath: /usr/share/elasticsearch/lib/spatial4j-0.5.jar;
> [2016-04-13T10:19:22.658075] Add path to classpath: /usr/share/elasticsearch/lib/t-digest-3.0.jar;
> [2016-04-13T10:19:22.731812] Add path to classpath: /opt/syslog-ng/lib/syslog-ng/java-modules/syslog-ng-core.jar;
> [2016-04-13T10:19:22.900320] Exception occured: java.lang.IllegalStateException: path.home is not configured
> at org.elasticsearch.env.Environment.<init>(Environment.java:101)
> at org.elasticsearch.node.internal.InternalSettingsPreparer.prepareEnvironment(InternalSettingsPreparer.java:81)
> at org.elasticsearch.node.Node.<init>(Node.java:140)
> at org.elasticsearch.node.NodeBuilder.build(NodeBuilder.java:143)
> at org.elasticsearch.node.NodeBuilder.node(NodeBuilder.java:150)
> at org.syslog_ng.elasticsearch_v2.client.ESNodeClient.createClient(ESNodeClient.java:78)
> at org.syslog_ng.elasticsearch_v2.client.ESClient.init(ESClient.java:100)
> at org.syslog_ng.elasticsearch_v2.ElasticSearchDestination.init(ElasticSearchDestination.java:63)
> at org.syslog_ng.LogPipe.initProxy(LogPipe.java:64)
>
>
>> On Apr 13, 2016, at 8:53 AM, Fabien Wernli <wernli at in2p3.fr> wrote:
>>
>> On Wed, Apr 13, 2016 at 08:48:47AM -0400, Scot wrote:
>>> How do I specify ?
>>
>> there's two destinations: elasticsearch and elasticsearch_v2
>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20160413/0d98533e/attachment-0001.htm
More information about the syslog-ng
mailing list