[syslog-ng] attempting to get parsed logs into elasticsearch

[Russell Fulton]

Now that I have the path to the ES library sorted *and* got the firewall rules sorted I am getting the following errors:

Starting syslog-ng
/usr/lib/jvm/java-1.7.0-oracle-1.7.0.91.x86_64/jre/lib/amd64/server
[2016-09-30T14:38:36.419846] Can't get name by unique options;
[2016-09-30T14:38:36.419914] Name by uniq options is empty;
[2016-09-30T14:38:36.419923] Error initializing message pipeline;

I am guessing that I need to set up something more on the ES end?
 
Russell

> On 30 Sep 2016, at 12:56, Russell Fulton <r.fulton at auckland.ac.nz> wrote:
> 
> Hi folks
> 
> I am trying to get some parsed logs into elasticssearch but have ended up with a cryptic error message:
> 
> Starting syslog-ng
> /usr/lib/jvm/java-1.7.0-oracle-1.7.0.91.x86_64/jre/lib/amd64/server
> [2016-09-30T12:43:43.649899] Error initializing message pipeline;
> 
> which almost certainly relates to the ES set up but I have no idea what is actually wrong.  The ES logs do not show anything.
> 
> Config file:
> @version: 3.8
> @module mod-java
> @include "scl.conf"
> 
> 
> options {
>        use_dns (no);
>        use_fqdn (no);
>        keep_hostname (yes);
> };
> 
> 
> source s_loghost {
>    tcp(flags(no-multi-line) port(1514) keep-alive(yes));
> };
> 
> destination d_syslog { file("/var/log/syslog.log"); };
> 
> destination d_elastic {
>  elasticsearch(
>    index("auth_${YEAR}.${MONTH}.${DAY}")
>    type("auth")
>    cluster("security")
>    flush-limit("1000")
>  );
> };
> 
> parser p_patterns {  db-parser( file("/etc/syslog-ng/merged.xml")); };
> 
> log {
> 
>    source(s_loghost);
>    parser (p_patterns);
>    destination(d_elastic );
> 
> };
> 
> The same configuration with a json file destination works fine.
> 
> Any hints on what to look at appreciated.
> 
> The ES instance running on the host is set to data: no and I expect it to ship the data to one of the other nodes which has storage.
> 
> Russell (who admits to being an ES novice)
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>