[syslog-ng] attempting to get parsed logs into elasticsearch

[Russell Fulton]

Thanks Scot! that fixed that particular issue.  I will try on the next one now :)

Russell

> On 30 Sep 2016, at 13:18, Scot <scotrn at gmail.com> wrote:
> 
> Anytime I had this error is was in java library path for the module. 
> 
> Try adding client_lib_dir("/opt/elasticsearch/lib") to your d_elastic destination.  
> With the right path to es libs. 
> 
> 
> 
> 
> On Thu, Sep 29, 2016 at 7:56 PM, Russell Fulton <r.fulton at auckland.ac.nz <mailto:r.fulton at auckland.ac.nz>> wrote:
> Hi folks
> 
> I am trying to get some parsed logs into elasticssearch but have ended up with a cryptic error message:
> 
> Starting syslog-ng
> /usr/lib/jvm/java-1.7.0-oracle-1.7.0.91.x86_64/jre/lib/amd64/server
> [2016-09-30T12:43:43.649899] Error initializing message pipeline;
> 
> which almost certainly relates to the ES set up but I have no idea what is actually wrong.  The ES logs do not show anything.
> 
> Config file:
> @version: 3.8
> @module mod-java
> @include "scl.conf"
> 
> 
> options {
>         use_dns (no);
>         use_fqdn (no);
>         keep_hostname (yes);
> };
> 
> 
> source s_loghost {
>     tcp(flags(no-multi-line) port(1514) keep-alive(yes));
> };
> 
> destination d_syslog { file("/var/log/syslog.log"); };
> 
> destination d_elastic {
>   elasticsearch(
>     index("auth_${YEAR}.${MONTH}.${DAY}")
>     type("auth")
>     cluster("security")
>     flush-limit("1000")
>   );
> };
> 
> parser p_patterns {  db-parser( file("/etc/syslog-ng/merged.xml")); };
> 
> log {
> 
>     source(s_loghost);
>     parser (p_patterns);
>     destination(d_elastic );
> 
> };
> 
> The same configuration with a json file destination works fine.
> 
> Any hints on what to look at appreciated.
> 
> The ES instance running on the host is set to data: no and I expect it to ship the data to one of the other nodes which has storage.
> 
> Russell (who admits to being an ES novice)
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng <https://lists.balabit.hu/mailman/listinfo/syslog-ng>
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng <http://www.balabit.com/support/documentation/?product=syslog-ng>
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq <http://www.balabit.com/wiki/syslog-ng-faq>
> 
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20160930/bcba4207/attachment.htm