I think all the TCP port connections are correct it’s just a configuration to get ES to store data. [root@loghost etc]# wget http://localhost:9200 --2016-04-14 09:37:48-- http://localhost:9200/ Resolving localhost (localhost)... ::1, 127.0.0.1 Connecting to localhost (localhost)|::1|:9200... connected. HTTP request sent, awaiting response... 200 OK Length: 310 [application/json] Saving to: ‘index.html’ 100%[=========================================================================================================>] 310 --.-K/s in 0s 2016-04-14 09:37:48 (20.9 MB/s) - ‘index.html’ saved [310/310]
On Apr 14, 2016, at 8:33 AM, Jim Hendrick <jrhendri@roadrunner.com> wrote:
Just for grins, try querying ES directly with curl - it may be that kibana isn't configured to show the right type//index/thingy...
had some issues like this with the old Kibana 4 a year or so ago - haven't poked at it in a little while though.
jim
On 04/14/2016 01:37 AM, Scot Needy wrote:
Hi,
When setting up syslog-ng -> ELK the logstash portion should not be needed as syslog-ng writes directly to an ES node or remote “transport”
My understanding is logstash would post parse a log for a given interval and send it to ES like lodrotated. Not a solution for a syslog-ng realtime model. So my ElK stack is built but I don’t seem to be able to make the connection or data format between syslog-ng 3.8 and ES2. I can see syslog-ng writing data to the local log file destination and syslog-ng and es2 logs indicate the TCP connection was successful on startup… But I see no data in Kibana nor do I see any activity in syslog-ng or es logs using either of the templates below.
NOTE: Removing syslog-ng->es and running flat logs through logstash does seem to populate data but that is not the solution.
destination d_es { elasticsearch2( index("syslog-ng_${YEAR}.${MONTH}.${DAY}") type("syslog-ng") # Description: The type of the index. For example, type("test") template("$(format-json --scope rfc5424 --exclude DATE --key ISODATE @timestamp=${ISODATE})") #template("$(format-json --scope rfc3164 --scope nv-pairs --exclude R_DATE --key ISODATE)\n")
port("9300") server("localhost") flush_limit("5000") client_mode("node") cluster("syslog-ng") custom_id("syslog") resource("/etc/elasticsearch/elasticsearch.yml") client_lib_dir("/usr/share/elasticsearch/lib") concurrent_requests("100") ); };
# sending logs to ES destination log { source(s_net); parser(pattern_db); destination(d_es); flags(flow-control); };
/etc/elasticsearch/elasticsearch.yml cluster.name: syslog node.name: node-1 path.home: /usr/share/elasticsearch ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq