Hi all,

As you may have all known already, We are going to develop a new feature, Kafka source for syslog-ng product during this GSoC 2016 under the guidance of Viktor Juhasz.

A simple abstract idea of this project is,

The syslog-ng application can read messages from the sources. It processes them with filters, rewrite rules, parsers and finally sends messages to their destinations. The syslog-ng application already has a Kafka destination that is implemented in Java. The Kafka source will allow syslog-ng to read messages from Kafka, for an example this can be used for example as a queue between several syslog-ng instances.

Kafka source project will allow you to,
-Use Kafka consumer which uses the high level Kafka group consuming API.

-Continue reading from the last message from which syslog-ng reloaded/restarted.
-Avoid message loosing or message duplication as much as possible whenever there is multiple syslog-ng reading the same kafka input.

If you are interested to become a feature owner for Kafka source project and give us guidance on user's perspective, it will be very easy for us to build a effective component. Appreciate if you can contact us by sending mail to us or to this thread.

Thanks,

Best regards,

Vithulan.

Vithulan MV.

Undergraduate,

Department of Computer Science & Engineering,

University of Moratuwa,

Sri lanka.