Hi, Yes, that's right. Just seeing syslog-ng-java-deps installed I assumed that you skipped that step. Bye, Peter Czanik (CzP) <peter.czanik@balabit.com> Balabit / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik On Fri, Aug 17, 2018 at 11:27 AM, Lee Keng Ket <kengket@gmail.com> wrote:
Hi, Peter
I have put in all the jar files from the Hadoop client to the /opt/hadoop/lib and declare this in the conf file. This should solve the JAR dependencies, right? Or please correct me if I'm wrong.
Thank you.
Regards, KK
On Fri, Aug 17, 2018 at 4:34 PM Czanik, Péter <peter.czanik@balabit.com> wrote:
Hi,
As https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng314/ also writes: "Java is enabled, but JAR dependencies are not provided in package, except for Elasticsearch http mode." The syslog-ng-java-deps.noarch contains build time dependencies. Probably I should rename the package to syslog-ng-java-build-deps...
Check the documentation at https://www.syslog-ng.com/ technical-documents/doc/syslog-ng-open-source-edition/ 3.16/administration-guide/33#TOPIC-956506 on how to download and configure HDFS related JAR dependencies.
Bye,
Peter Czanik (CzP) <peter.czanik@balabit.com> Balabit / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik
On Fri, Aug 17, 2018 at 10:22 AM, Lee Keng Ket <kengket@gmail.com> wrote:
Hi,
I'm trying to connect syslog-ng 3.14.1 to HDFS to store the syslog messages. The syslog-ng can start without error, and it's able to write into local file. However, the log is not written to the HDFS. As there is no single error, I'm not sure how I should troubleshoot on this.
I have installed the syslog-ng from this repo, https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng314/ repo/epel-7/czanik-syslog-ng314-epel-7.repo
Installed Packages syslog-ng.x86_64 3.14.1-4.el7.centos @czanik-syslog-ng314 syslog-ng-java.x86_64 3.14.1-4.el7.centos @czanik-syslog-ng314 syslog-ng-java-deps.noarch 1.0-2 @czanik-syslog-ng314
This is the message from /var/log/message: Log statistics; processed='src.internal(s_sys#0)=1', stamp='src.internal(s_sys#0)=1534491834', processed='destination(d_spol)=0', processed='destination(d_mlal)=0', processed='center(received)=2', processed='destination(d_mesg)=1', processed='destination(d_mail)=0', processed='destination(d_auth)=0', processed='destination(d_cron)=0', processed='destination(d_hdfs)=1', processed='center(queued)=3', queued='global(scratch_buffers_count)=0', processed='source(remote_log)=1', dropped='dst.java(d_hdfs#0,java_dst,hdfs,hdfs://x.x.x.x: 25000,/user/syslog/$HOST-$DAY-$MONTH-$YEAR.log)=0', processed='dst.java(d_hdfs#0,java_dst,hdfs,hdfs://x.x.x.x: 25000,/user/syslog/$HOST-$DAY-$MONTH-$YEAR.log)=1', queued='dst.java(d_hdfs#0,java_dst,hdfs,hdfs://x.x.x.x: 25000,/user/syslog/$HOST-$DAY-$MONTH-$YEAR.log)=1', processed='global(payload_reallocs)=0', processed='src.journald(journal)=0', stamp='src.journald(journal)=0', processed='global(sdata_updates)=0', queued='global(scratch_buffers_bytes)=0', processed='destination(d_boot)=0', processed='destination(d_kern)=0', processed='source(s_sys)=1', processed='destination(remote)=1', processed='global(internal_queue_length)=0', processed='global(msg_clones)=0'
Anyone has any idea how should I proceed the troubleshooting?
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq