On 30 Sep 2016, at 13:18, Scot <scotrn@gmail.com> wrote:

Anytime I had this error is was in java library path for the module. 

Try adding client_lib_dir("/opt/elasticsearch/lib") to your d_elastic destination.  

With the right path to es libs. 

On Thu, Sep 29, 2016 at 7:56 PM, Russell Fulton <r.fulton@auckland.ac.nz> wrote:

Hi folks

I am trying to get some parsed logs into elasticssearch but have ended up with a cryptic error message:

Starting syslog-ng
/usr/lib/jvm/java-1.7.0-oracle-1.7.0.91.x86_64/jre/lib/amd64/server
[2016-09-30T12:43:43.649899] Error initializing message pipeline;

which almost certainly relates to the ES set up but I have no idea what is actually wrong.  The ES logs do not show anything.

Config file:
@version: 3.8
@module mod-java
@include "scl.conf"


options {
        use_dns (no);
        use_fqdn (no);
        keep_hostname (yes);
};


source s_loghost {
    tcp(flags(no-multi-line) port(1514) keep-alive(yes));
};

destination d_syslog { file("/var/log/syslog.log"); };

destination d_elastic {
  elasticsearch(
    index("auth_${YEAR}.${MONTH}.${DAY}")
    type("auth")
    cluster("security")
    flush-limit("1000")
  );
};

parser p_patterns {  db-parser( file("/etc/syslog-ng/merged.xml")); };

log {

    source(s_loghost);
    parser (p_patterns);
    destination(d_elastic );

};

The same configuration with a json file destination works fine.

Any hints on what to look at appreciated.

The ES instance running on the host is set to data: no and I expect it to ship the data to one of the other nodes which has storage.

Russell (who admits to being an ES novice)
______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq