Perfect! I should have seen it before. When I was sending logs and I was not seeing anything in Kibana I thought something was wrong (even captured the packets and it showed the whole message actually being sent) Today I finally noticed I was using the "logstash" Kibana dashboard and when I switched to the generic one all the parsed data was there. I still need to get it built on a more production system - but I'm sure that will go OK once I spend some time. Thanks! Jim On 10/23/2014 11:03 AM, Fabien Wernli wrote:
On Thu, Oct 23, 2014 at 10:50:55AM -0400, jrhendri@roadrunner.com wrote:
Are you saying I would not need to use the format-json bit? If so - how would I select/name the desired fields that were parsed with patterndb? By simply passing `scope` to the destination block [1] I also use a special `exclude` [2] parameter that lets me furter drop unwanted name-values.
As far as overall performance - I really think it is a combination of disk I/O and memory starvation. I'm using collectd, riemann and riemann-dash to monitor syslog-ng and ES performance live
[1] https://github.com/faxm0dem/syslog_ng-elasticsearch/blob/master/perl/syslog-... [2] https://github.com/faxm0dem/syslog_ng-elasticsearch/blob/master/perl/plugin.... ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq