Hi,

I'm trying to connect syslog-ng 3.14.1 to HDFS to store the syslog messages. The syslog-ng can start without error, and it's able to write into local file. However, the log is not written to the HDFS. As there is no single error, I'm not sure how I should troubleshoot on this.

I have installed the syslog-ng from this repo, https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng314/repo/epel-7/czanik-syslog-ng314-epel-7.repo

Installed Packages
syslog-ng.x86_64                                                                        3.14.1-4.el7.centos                                                              @czanik-syslog-ng314
syslog-ng-java.x86_64                                                                   3.14.1-4.el7.centos                                                              @czanik-syslog-ng314
syslog-ng-java-deps.noarch                                                              1.0-2                                                                            @czanik-syslog-ng314

This is the message from /var/log/message:
Log statistics; processed='src.internal(s_sys#0)=1', stamp='src.internal(s_sys#0)=1534491834', processed='destination(d_spol)=0', processed='destination(d_mlal)=0', processed='center(received)=2', processed='destination(d_mesg)=1', processed='destination(d_mail)=0', processed='destination(d_auth)=0', processed='destination(d_cron)=0', processed='destination(d_hdfs)=1', processed='center(queued)=3', queued='global(scratch_buffers_count)=0', processed='source(remote_log)=1', dropped='dst.java(d_hdfs#0,java_dst,hdfs,hdfs://x.x.x.x:25000,/user/syslog/$HOST-$DAY-$MONTH-$YEAR.log)=0', processed='dst.java(d_hdfs#0,java_dst,hdfs,hdfs://x.x.x.x:25000,/user/syslog/$HOST-$DAY-$MONTH-$YEAR.log)=1', queued='dst.java(d_hdfs#0,java_dst,hdfs,hdfs://x.x.x.x:25000,/user/syslog/$HOST-$DAY-$MONTH-$YEAR.log)=1', processed='global(payload_reallocs)=0', processed='src.journald(journal)=0', stamp='src.journald(journal)=0', processed='global(sdata_updates)=0', queued='global(scratch_buffers_bytes)=0', processed='destination(d_boot)=0', processed='destination(d_kern)=0', processed='source(s_sys)=1', processed='destination(remote)=1', processed='global(internal_queue_length)=0', processed='global(msg_clones)=0'

Anyone has any idea how should I proceed the troubleshooting?

______________________________________________________________________________
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
FAQ: http://www.balabit.com/wiki/syslog-ng-faq