Hi Vincent, the currently available Java features are * Kafka * Elasticsearch * HDFS (we have HTTP but we also have curl destination in C). As you mentioned we have Kafka destination driver written in C in syslog-ng incubator, so users can use it. It should not be a problem if a feature is available only in the syslog-ng incubator. As the Kafka C client is stable enough, I don't see any problems in integrating it to the syslog-ng master (just someone should create a pull request). In case of Elasticsearch we are happy that we are able to support even the latest Elasticsearch 5 but we don't have the resource to re-implement it in C after every new Elasticsearch release. If someone wants to implement a REST Elasticsearch client in C and based on that also implements an ES dest. driver in C then, we will happily integrate it into syslog-ng. HDFS: users who uses HDFS will have Java on their machines, so it should not be a problem. I think a feature implemented in Java is much-much better than an unimplemented feature... regards, Laszlo Budai Sent from my HTC ----- Reply message ----- From: "Vincent Bernat" <bernat@luffy.cx> To: "Vithulan MV" <vithulanmv.12@cse.mrt.ac.lk> Cc: "Syslog-ng users' and developers' mailing list" <syslog-ng@lists.balabit.hu> Subject: [syslog-ng] Finding feature owner for Kafka source project Date: Sat, May 21, 2016 19:38 ❦ 21 mai 2016 20:33 +0530, Vithulan MV <vithulanmv.12@cse.mrt.ac.lk> :

A simple abstract idea of this project is, The syslog-ng application can read messages from the sources. It processes them with filters, rewrite rules, parsers and finally sends messages to their destinations. The syslog-ng application already has a Kafka destination that is implemented in Java. The Kafka source will allow syslog-ng to read messages from Kafka, for an example this can be used for example as a queue between several syslog-ng instances.

It's a bit annoying that so many interesting features have to go through Java. One of the major point of syslog-ng is its light footprint. librdkafka has support for both source and destination and is officially supported by Confluent. Its development matches Kafka one and it supports every feature in Kafka 0.9, including SSL and the new consumer. It's a bit late to complain, notably if you chose this project because you were comfortable with Java (and your mentor is obviously interested in doing that in Java too). However, this has already been pointed out in March without much response from Balabit. It's not just a matter of proposing code since such code is likely to get stuck in the incubator forever. -- Make sure special cases are truly special. - The Elements of Programming Style (Kernighan & Plauger) ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq