Anytime I had this error is was in java library path for the module. Try adding client_lib_dir("/opt/elasticsearch/lib") to your d_elastic destination. With the right path to es libs. On Thu, Sep 29, 2016 at 7:56 PM, Russell Fulton <r.fulton@auckland.ac.nz> wrote:

Hi folks

I am trying to get some parsed logs into elasticssearch but have ended up with a cryptic error message:

Starting syslog-ng /usr/lib/jvm/java-1.7.0-oracle-1.7.0.91.x86_64/jre/lib/amd64/server [2016-09-30T12:43:43.649899] Error initializing message pipeline;

which almost certainly relates to the ES set up but I have no idea what is actually wrong. The ES logs do not show anything.

Config file: @version: 3.8 @module mod-java @include "scl.conf"

options { use_dns (no); use_fqdn (no); keep_hostname (yes); };

source s_loghost { tcp(flags(no-multi-line) port(1514) keep-alive(yes)); };

destination d_syslog { file("/var/log/syslog.log"); };

destination d_elastic { elasticsearch( index("auth_${YEAR}.${MONTH}.${DAY}") type("auth") cluster("security") flush-limit("1000") ); };

parser p_patterns { db-parser( file("/etc/syslog-ng/merged.xml")); };

log {

source(s_loghost); parser (p_patterns); destination(d_elastic );

};

The same configuration with a json file destination works fine.

Any hints on what to look at appreciated.

The ES instance running on the host is set to data: no and I expect it to ship the data to one of the other nodes which has storage.

Russell (who admits to being an ES novice) ____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

Now that I have the path to the ES library sorted *and* got the firewall rules sorted I am getting the following errors: Starting syslog-ng /usr/lib/jvm/java-1.7.0-oracle-1.7.0.91.x86_64/jre/lib/amd64/server [2016-09-30T14:38:36.419846] Can't get name by unique options; [2016-09-30T14:38:36.419914] Name by uniq options is empty; [2016-09-30T14:38:36.419923] Error initializing message pipeline; I am guessing that I need to set up something more on the ES end? Russell

On 30 Sep 2016, at 12:56, Russell Fulton <r.fulton@auckland.ac.nz> wrote:

Hi folks

I am trying to get some parsed logs into elasticssearch but have ended up with a cryptic error message:

Starting syslog-ng /usr/lib/jvm/java-1.7.0-oracle-1.7.0.91.x86_64/jre/lib/amd64/server [2016-09-30T12:43:43.649899] Error initializing message pipeline;

which almost certainly relates to the ES set up but I have no idea what is actually wrong. The ES logs do not show anything.

Config file: @version: 3.8 @module mod-java @include "scl.conf"

options { use_dns (no); use_fqdn (no); keep_hostname (yes); };

source s_loghost { tcp(flags(no-multi-line) port(1514) keep-alive(yes)); };

destination d_syslog { file("/var/log/syslog.log"); };

destination d_elastic { elasticsearch( index("auth_${YEAR}.${MONTH}.${DAY}") type("auth") cluster("security") flush-limit("1000") ); };

parser p_patterns { db-parser( file("/etc/syslog-ng/merged.xml")); };

log {

source(s_loghost); parser (p_patterns); destination(d_elastic );

};

The same configuration with a json file destination works fine.

Any hints on what to look at appreciated.

The ES instance running on the host is set to data: no and I expect it to ship the data to one of the other nodes which has storage.

Russell (who admits to being an ES novice) ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq