Hi, I don't use Kibana regularly, but have some distant memories: in the upper right corner there is a "settings" icon. Once you click on it, "index pattern" will appear in the upper left corner with a pencil icon next to it. Click on it, and you will have an orange "reload field list" icon at the top of the screen. (this is with version 4.0) Bye, Peter Czanik (CzP) <peter.czanik@balabit.com> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik On Thu, Sep 10, 2015 at 11:00 PM, <jrhendri@roadrunner.com> wrote:
Hi, I am testing elasticsearch with the 3.7.1 ose build on ubuntu 14.04 and have some questions regarding how to get elasticsearch & kibana to "see" the individual fields within a structured syslog message.
I have tried a few different formats but all the >key>=<value> pairs appear within the MESSAGE part. For example:
MESSAGE 2015-09-09T17:00:06.775 0055-inet-fw-node0 RT_FLOW - RT_FLOW_SESSION_CREATE_LS [junos@2636.1.1.1.2.35 logical-system-name="internetVR" source-address="143.115.190.50" source-port="42241" destination-address="70.39.233.137" destination-port="53" service-name="junos-dns-udp" nat-source-address="143.115.190.50" nat-source-port="42241" nat-destination-address="70.39.233.137" nat-destination-port="53" src-nat-rule-name="None" dst-nat-rule-name="None" protocol-id="17" policy-name="Device-Zone-903" source-zone-name="dns-b2b" destination-zone-name="internet" session-id-32="80968105" username="N/A" roles="N/A" packet-incoming-interface="reth3.120" application="UNKNOWN" nested-application="UNKNOWN" encrypted="UNKNOWN"]
The "Available Fields" I see in kibana are:
@timestamp tDATE tFACILITY tHOST tMESSAGE tPRIORITY tPROGRAM t_id t_index t_type
I would like to be able to have each field recognized separately so that kibana could search for specific things like "source-address", etc. I thought these would be available under the .SDATA. set, but apparently I missed something.
Is this possible (and I am just lacking understanding) or am I expecting too much?
These are the last three tests I have used within the elasticsearch destination and they all essentially result in the same thing.
#!# option( "message_template", "$(format-json --scope nv_pairs)\n") #!# option( "message_template", "$(format-json --scope rfc5424 --exclude DATE --key ISODATE @timestamp=${ISODATE})\n") option( "message_template", "$(format-json --scope rfc5424 @timestamp=${ISODATE} --key .SDATA.* ) \n" )
Thanks for any help or guidance!
Jim
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq