[syslog-ng] syslog-ng to elasticsearch

[Budai, László]

Hi,

3.7.2 does not work with ES 2.2 (due to some internal API change between
ES2.1 and 2.2).

Maybe a different issue, but you could take a look at:
https://github.com/balabit/syslog-ng/issues/967

and another related github issue you can follow:
https://github.com/balabit/syslog-ng/issues/970


regards,
Laszlo Budai

On Fri, Mar 4, 2016 at 12:38 PM, Mike Lewis <MLewis at nephilaadvisors.co.uk>
wrote:

> Hi,
>
>
>
> I’m having some issues trying to setup (syslog-ng v3.7.2) an elastic
> search destination. ES 2.2.0.
>
>
>
> In my syslog-ng.conf file, I have the destination defined as:
>
>
>
> destination d_elasticsearch {
>
>      elasticsearch(
>
>          index("syslog-ng_${YEAR}.${MONTH}.${DAY}")
>
>          type("syslog-ng")
>
>
>          class-path("/usr/lib64/syslog-ng/java-modules/*.jar:/usr/lib/syslog-ng-java-module-dependency-jars/jars/*.jar:/usr/share/elasticsearch/lib/*.jar:/usr/share/elasticsearch/modules/*.jar")
>
>           client_mode("transport")
>
>          server("172.16.100.137")
>
>          port("9300")
>
>          cluster("dev-elasticsearch")
>
>           template("$(format-json -s all-nv-pairs -p @timestamp=$ISODATE
> -p @message=$MSG)")
>
>      );
>
>  };
>
>
>
>
>
> However, in the elastic search logs, I just see an exception through on
> each connection attempt:
>
>
>
>
>
> [2016-03-04 06:33:00,737][WARN ][transport.netty          ] [node-1]
> exception caught on transport layer [[id: 0xe12086b7, /
> 172.16.100.137:52583 => /172.16.100.137:9300]], closing connection
>
> java.lang.IllegalStateException: Message not fully read (request) for
> requestId [0], action [cluster/state], readerIndex [34] vs expected [49];
> resetting
>
>         at
> org.elasticsearch.transport.netty.MessageChannelHandler.messageReceived(MessageChannelHandler.java:120)
>
>
>
> Has anyone come across this issues previously?
>
>
>
> Regards,
>
> Mike Lewis
>
>
>
> --------------------------------------------------------------------------------------------------------------------------
>
> This email has been sent to you on behalf of Nephila Advisors LLC
> (“Advisors”). Advisors provides consultancy services to Nephila Capital
> Ltd. (“Capital”), an investment advisor managed and carrying on business in
> Bermuda. Advisors and its employees do not act as agents for Capital or the
> funds it advises and do not have the authority to bind Capital or such
> funds to any transaction or agreement.
>
> The information in this e-mail, and any attachment therein, is
> confidential and for use by the addressee only. Any use, disclosure,
> reproduction, modification or distribution of the contents of this e-mail,
> or any part thereof, other than by the intended recipient, is strictly
> prohibited. If you are not the intended recipient, please return the e-mail
> to the sender and delete it from your computer. This email is for
> information purposes only, nothing contained herein constitutes an offer to
> sell or buy securities, as such an offer may only be made from a properly
> authorized offering document. Although Nephila attempts to sweep e-mail and
> attachments for viruses, it does not guarantee that either are virus-free
> and accepts no liability for any damage sustained as a result of viruses.
> --------------------------------------------------------------------------------------------------------------------------
>
>
> --------------------------------------------------------------------------------------------------------------------------
>
> This email has been sent to you on behalf of Nephila Advisors UK
> (“Advisors UK”). Advisors UK provides consultancy services to Nephila
> Capital Ltd. (“Capital”), an investment advisor managed and carrying on
> business in Bermuda. Advisors UK and its employees do not act as agents for
> Capital or the funds it advises and do not have the authority to bind
> Capital or such funds to any transaction or agreement.
>
> The information in this e-mail, and any attachment therein, is
> confidential and for use by the addressee only. Any use, disclosure,
> reproduction, modification or distribution of the contents of this e-mail,
> or any part thereof, other than by the intended recipient, is strictly
> prohibited. If you are not the intended recipient, please return the e-mail
> to the sender and delete it from your computer. This email is for
> information purposes only, nothing contained herein constitutes an offer to
> sell or buy securities, as such an offer may only be made from a properly
> authorized offering document. Although Nephila attempts to sweep e-mail and
> attachments for viruses, it does not guarantee that either are virus-free
> and accepts no liability for any damage sustained as a result of viruses.
> --------------------------------------------------------------------------------------------------------------------------
>
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20160304/832a2cca/attachment-0001.htm