Hi all,

First of all, I'd like to thank you Syslog-ng community and my mentor Viktor Juhasz for accepting my proposal to implement Kafka source in Java!.
Some of you may already know me but for others, I'll introduce myself again. I'm Vithulan, 3rd year undergraduate at University of Moratuwa, Sri lanka. My areas of interests are, Data analytics, Machine Learning, Networking and Security. 

As a first step into this GSoC program I already started following the documentation [1], and going through the code base of syslog-ng (I'm curious what IDE you all are using?). What are the other areas that you want me to know to polish my knowledge before we get into the coding period?

I guess, this is going to be very interesting 4 months with good new experiences :)
Thank you!

[1] https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/preface.html

Best regards,
Vithulan.

On 23 April 2016 at 08:42, Vithulan MV <vithulanmv.12@cse.mrt.ac.lk> wrote:
Hi Viktor,

I am very excited and happy after seeing my morning mail from Google summer of code! Thank you very much for accepting my proposal! :)
How shall we proceed into this project? and any tips before going deep into the project? :) 

Thank you.

Best regards,
Vithulan.

On 26 March 2016 at 10:52, Vithulan MV <vithulanmv.12@cse.mrt.ac.lk> wrote:

Hi Viktor,

I have submitted my proposal in GSoC page and Syslog-ng wiki page aswell.

Thanks.
Best regards,
Vithulan

On Mar 10, 2016 8:42 PM, "Vithulan MV" <vithulanmv.12@cse.mrt.ac.lk> wrote:
Hi Viktor,

Thanks for the link :) I'll check that.

Regards,
Vithulan.

On 10 March 2016 at 20:22, Juhász, Viktor <viktor.juhasz@balabit.com> wrote:
Hi,

Sorry I missed the link :)
https://syslog-ng.org/gsoc-2016/

BR,
Viktor

On Thu, Mar 10, 2016 at 3:46 PM, Vithulan MV <vithulanmv.12@cse.mrt.ac.lk> wrote:
Hi Viktor,

Do you mean creating proposals for GSOC 2016 page [1] ? If not please kindly share the page link with me?

[1] https://github.com/balabit/syslog-ng/wiki/Creating-a-proposal-for-GSoC-2016

Thanks.
Regards,
Vithulan.

On 10 March 2016 at 19:40, Juhász, Viktor <viktor.juhasz@balabit.com> wrote:
Hi Vithulan,

We've finished our GSoC process description. Please read it carefully, and fill our Student Application Form.

BR,
Viktor

On Thu, Mar 10, 2016 at 3:06 PM, Vithulan MV <vithulanmv.12@cse.mrt.ac.lk> wrote:
HI Viktor,

Thanks for the overview and giving the overall idea. I'm working on this and report you back on the progress.

Thanks.
Regards,
Vithulan.

On 8 March 2016 at 18:05, Juhász, Viktor <viktor.juhasz@balabit.com> wrote:
Hi Vithulan,

Yes, this should be plugin (you can find the plugins in the modules directory in syslog-ng source).

Just a very briefly overview:

syslog-ng read messages from the sources, process them with filters, rewrite rules, parsers, etc, and finally send messages to the destinations.
We have a Kafka destination, it is implemented in java. The kafka source also should be implemented in java, but we our java source base is under construction, so till I can't give you an exact API, it is comming in the next 2-3 weeks.

The ovarall idea is the following.
 - We should implement a Kafka consumer which uses the high level kafka group consuming API (https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example)
 - We have to process data read from the Kafka (create LogMessage from this. LogMessage is an internal struct in syslog-ng representing the in the software)
 - After reload or restart syslog-ng have to be able to continue reading messages from the last read message.
 - If there are more syslog-ng's reading the same Kafka input (these syslog-ng's have the same group name) avoid message loosing or message duplication as much as possible

Regards,
Viktor



On Mon, Mar 7, 2016 at 6:43 PM, Vithulan MV <vithulanmv.12@cse.mrt.ac.lk> wrote:

Hi Viktor,

Oh I see! I've sent a mail to the mail list. Can you kindly give me an overall idea of project Kafka source please? (Architecture of syslog-ng/ current APIs) Is this is going to be an extension of syslog-ng?

Thank you.

Regards,
Vithulan.

Hi Vithulan,

I've never faced with this problem, during compiling syslog-ng.
I suggest, that ask about your problem in the mail list of syslog-ng ;)

Best Regards,
Viktor


On Mon, Mar 7, 2016 at 12:46 PM, Vithulan MV <vithulanmv.12@cse.mrt.ac.lk> wrote:
Hi,

I've been going through the git book that you referred. It was helpful. Can you please explain a bit about the project structure and deliverables of Kafka-source project?

I added required dependencies and tried to build syslog-ng from the source, there was an error saying,
${syslog-ng_HOME}/modules/afmongodb/mongo-c-driver/src/mongoc/mongoc-scram.c: In function '_mongoc_scram_sha1': storage size of 'digest_ctx' isn't known (EVP_MD_CTX digest_ctx)
I confirmed OpenSSL [1] headers are included in the file.Can you please point out whats wrong?

In the meantime I'm going through the syslog-ng documentation [2]. Can you give me directives on what I can be doing while preparing my proposals?
Thank you.
Regards,
Vithulan.


On 4 March 2016 at 18:56, Vithulan MV <vithulanmv.12@cse.mrt.ac.lk> wrote:
Hi,

Thanks for the reference, I will go through it and report you back on the progress.

Thanks,
Regards,
Vithulan.  

On 4 March 2016 at 18:45, Juhász, Viktor <viktor.juhasz@balabit.com> wrote:
Hello,

Sorry for the long response time. We are working on a wiki page, this page will describe all the required information. Till please read our documentation (getting started) https://www.gitbook.com/book/syslog-ng/getting-started/details

Best Regards,
Viktor Juhasz

On Fri, Mar 4, 2016 at 2:09 PM, Vithulan MV <vithulanmv.12@cse.mrt.ac.lk> wrote:
Hello,

I sent the following mail to the syslog-ng mailing list, but didn't get any responses. So I thought it didn't reach to you. I'm highly motivated on contributing for syslog-ng in GSoC 2016. I also have prior experience in syslog-ng domain so I feel like this project is going to be very interesting. Please kindly guide me on how I should start?

Thank you.

Regards,
Vithulan.


---------- Forwarded message ----------
From: Vithulan MV <vithulanmv.12@cse.mrt.ac.lk>
Date: 3 March 2016 at 15:33
Subject: [GSoC] Project: Kafka source
To: syslog-ng@lists.balabit.hu


Hi all,

I'm currently a 3rd year undergraduate at University of Moratuwa, Sri lanka. I have attached my Linkedin profile with this mail [1]. I'd like to contribute to a GSoC 2016 project with the open source community of syslog-ng.

I have contributed in log analyzing project LogAnalyzer [2] and also worked with logstash, kafka. Therefore, I think I can contribute syslog-ng project kafka source as I'm much interested in that.

Please kindly make any suggestions on how I should proceed on this?
Thank you.
Regards,
Vithulan

--
Vithulan MV.

Undergraduate, 
Department of Computer Science & Engineering,
University of Moratuwa,
Sri lanka.



--
Vithulan MV.

Undergraduate, 
Department of Computer Science & Engineering,
University of Moratuwa,
Sri lanka.




--
Vithulan MV.

Undergraduate, 
Department of Computer Science & Engineering,
University of Moratuwa,
Sri lanka.



--
Vithulan MV.

Undergraduate, 
Department of Computer Science & Engineering,
University of Moratuwa,
Sri lanka.





--
Vithulan MV.

Undergraduate, 
Department of Computer Science & Engineering,
University of Moratuwa,
Sri lanka.




--
Vithulan MV.

Undergraduate, 
Department of Computer Science & Engineering,
University of Moratuwa,
Sri lanka.




--
Vithulan MV.

Undergraduate, 
Department of Computer Science & Engineering,
University of Moratuwa,
Sri lanka.



--
Vithulan MV.

Undergraduate, 
Department of Computer Science & Engineering,
University of Moratuwa,
Sri lanka.



--
Vithulan MV.

Undergraduate, 
Department of Computer Science & Engineering,
University of Moratuwa,
Sri lanka.