I am running ES/Kibana 6.8.9-1 and am struggling with this issue. https://discuss.elastic.co/t/message-failed-to-find-message-in-kibana-logs/2... I have added my index to the *Logs Indices* field in the Logs configuration. When I look at the fields in a document I see a field called MESSAGE, but not message. I do not see a way to add this field in the configuration. Is it possible to have this document display in the Logs UI? Can I convert the fields in syslog-ng to lowercase before forwarding them to elastic? Thanks, Shawn
You can map it to lower case, but I am a bit surprised that this is required. Here's the definition of our elasticsearch-http destination: https://github.com/syslog-ng/syslog-ng/blob/master/scl/elasticsearch/elastic... Note the template () parameter which you can customize to include further mappings. Right now it only maps @timestamp to be the timestamp of the message. On Wed, May 27, 2020, 22:24 Shawn Taylor <staylor8@ncsu.edu> wrote:
I am running ES/Kibana 6.8.9-1 and am struggling with this issue.
https://discuss.elastic.co/t/message-failed-to-find-message-in-kibana-logs/2...
I have added my index to the *Logs Indices* field in the Logs configuration.
When I look at the fields in a document I see a field called MESSAGE, but not message.
I do not see a way to add this field in the configuration. Is it possible to have this document display in the Logs UI? Can I convert the fields in syslog-ng to lowercase before forwarding them to elastic?
Thanks,
Shawn
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Hi Shawn, On Wed, May 27, 2020 at 04:24:11PM -0400, Shawn Taylor wrote:
I am running ES/Kibana 6.8.9-1 and am struggling with this issue.
https://discuss.elastic.co/t/message-failed-to-find-message-in-kibana-logs/2...
I have added my index to the *Logs Indices* field in the Logs configuration.
When I look at the fields in a document I see a field called MESSAGE, but not message.
I do not see a way to add this field in the configuration. Is it possible to have this document display in the Logs UI? Can I convert the fields in syslog-ng to lowercase before forwarding them to elastic?
I don't use the "logs app" in Kibana, so I'm afraid I'm limited in my ability to help you. That being said, the thread you mention has been solved by changing the name of the message column by the user: You are right! My problem was that I was changing "message" field to "message_log", so really "message" field didn't exist. I have changed in Kibana Logs the "Log Columns" to add "message_logs" and it works now! So it seems you can change the name of the columns in kibana, and in your case, assuming you're using the default syslog-ng config, it should be MESSAGE.
Thanks Fabien, I can't seem to find this configuration option in Kibana. I see the MESSAGE field in the document, but I assume that it's case sensitive and doesn't recognize that field? Shawn On Thu, May 28, 2020 at 3:58 AM Fabien Wernli <wernli@in2p3.fr> wrote:
Hi Shawn,
On Wed, May 27, 2020 at 04:24:11PM -0400, Shawn Taylor wrote:
I am running ES/Kibana 6.8.9-1 and am struggling with this issue.
https://discuss.elastic.co/t/message-failed-to-find-message-in-kibana-logs/2...
I have added my index to the *Logs Indices* field in the Logs
configuration.
When I look at the fields in a document I see a field called MESSAGE, but not message.
I do not see a way to add this field in the configuration. Is it possible to have this document display in the Logs UI? Can I convert the fields in syslog-ng to lowercase before forwarding them to elastic?
I don't use the "logs app" in Kibana, so I'm afraid I'm limited in my ability to help you. That being said, the thread you mention has been solved by changing the name of the message column by the user:
You are right! My problem was that I was changing "message" field to "message_log", so really "message" field didn't exist. I have changed in Kibana Logs the "Log Columns" to add "message_logs" and it works now!
So it seems you can change the name of the columns in kibana, and in your case, assuming you're using the default syslog-ng config, it should be MESSAGE.
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Shawn Taylor Security Applications Technologies NC State University 1575 Varsity Drive Raleigh NC 27606 919.515.8507
Hi Shawn, On Thu, May 28, 2020 at 07:48:16AM -0400, Shawn Taylor wrote:
I can't seem to find this configuration option in Kibana. I see the MESSAGE field in the document, but I assume that it's case sensitive and doesn't recognize that field?
Yes, fields in lucene are case-sensitive (it's just JSON). You could of course change the field name to @message (I believe you need the @ char too) before sending it to ES. You can do this multiple ways, either by adding a rewrite rule in your logpath, or by changing the template in the elasticsearch destination as suggested by Bazsi. I suggest you read the documentation, or of course we can assist you further in this list.
participants (3)
-
Balazs Scheidler
-
Fabien Wernli
-
Shawn Taylor