<div dir="ltr">Hi all,<div><br></div><div>First of all, I'd like to thank you Syslog-ng community and my mentor Viktor Juhasz for accepting my proposal to implement Kafka source in Java!.</div><div>Some of you may already know me but for others, I'll introduce myself again. I'm Vithulan, 3rd year undergraduate at University of Moratuwa, Sri lanka. My areas of interests are, Data analytics, Machine Learning, Networking and Security. </div><div><br></div><div>As a first step into this GSoC program I already started following the documentation [1], and going through the code base of syslog-ng (I'm curious what IDE you all are using?). What are the other areas that you want me to know to polish my knowledge before we get into the coding period?</div><div><br></div><div>I guess, this is going to be very interesting 4 months with good new experiences :)</div><div>Thank you!</div><div><br></div><div>[1] <a href="https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/preface.html">https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/preface.html</a></div><div><br></div><div>Best regards,</div><div>Vithulan.</div></div><div class="gmail_extra"><br><div class="gmail_quote">On 23 April 2016 at 08:42, Vithulan MV <span dir="ltr"><<a href="mailto:vithulanmv.12@cse.mrt.ac.lk" target="_blank">vithulanmv.12@cse.mrt.ac.lk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Viktor,<div><br></div><div>I am very excited and happy after seeing my morning mail from Google summer of code! Thank you very much for accepting my proposal! :)</div><div>How shall we proceed into this project? and any tips before going deep into the project? :) </div><div><br></div><div>Thank you.</div><div><br></div><div>Best regards,</div><div>Vithulan.</div></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><div class="gmail_quote">On 26 March 2016 at 10:52, Vithulan MV <span dir="ltr"><<a href="mailto:vithulanmv.12@cse.mrt.ac.lk" target="_blank">vithulanmv.12@cse.mrt.ac.lk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">Hi Viktor,</p>
<p dir="ltr">I have submitted my proposal in GSoC page and Syslog-ng wiki page aswell. </p>
<p dir="ltr">Thanks.<br>
Best regards,<br>
Vithulan</p><div><div>
<div class="gmail_quote">On Mar 10, 2016 8:42 PM, "Vithulan MV" <<a href="mailto:vithulanmv.12@cse.mrt.ac.lk" target="_blank">vithulanmv.12@cse.mrt.ac.lk</a>> wrote:<br type="attribution"><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div>Hi Viktor,<br><br></div>Thanks for the link :) I'll check that. <br><br></div>Regards,<br></div>Vithulan.<br></div><div class="gmail_extra"><br><div class="gmail_quote">On 10 March 2016 at 20:22, Juhász, Viktor <span dir="ltr"><<a href="mailto:viktor.juhasz@balabit.com" target="_blank">viktor.juhasz@balabit.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi,<div><br></div><div>Sorry I missed the link :)<br><a href="https://syslog-ng.org/gsoc-2016/" target="_blank">https://syslog-ng.org/gsoc-2016/</a><br></div><div><br></div><div>BR,</div><div>Viktor</div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 10, 2016 at 3:46 PM, Vithulan MV <span dir="ltr"><<a href="mailto:vithulanmv.12@cse.mrt.ac.lk" target="_blank">vithulanmv.12@cse.mrt.ac.lk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Hi Viktor,<br><br></div>Do you mean creating proposals for GSOC 2016 page [1] ? If not please kindly share the page link with me?<br><br>[1] <a href="https://github.com/balabit/syslog-ng/wiki/Creating-a-proposal-for-GSoC-2016" target="_blank">https://github.com/balabit/syslog-ng/wiki/Creating-a-proposal-for-GSoC-2016</a><br><br></div>Thanks.<br></div>Regards,<br></div>Vithulan.<br></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On 10 March 2016 at 19:40, Juhász, Viktor <span dir="ltr"><<a href="mailto:viktor.juhasz@balabit.com" target="_blank">viktor.juhasz@balabit.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Vithulan,<div><br></div><div>We've finished our GSoC process description. Please read it carefully, and fill our Student Application Form.</div><div><br></div><div>BR,</div><div>Viktor</div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Thu, Mar 10, 2016 at 3:06 PM, Vithulan MV <span dir="ltr"><<a href="mailto:vithulanmv.12@cse.mrt.ac.lk" target="_blank">vithulanmv.12@cse.mrt.ac.lk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>HI Viktor,<br><br></div>Thanks for the overview and giving the overall idea. I'm working on this and report you back on the progress. <br><br></div>Thanks.<br></div>Regards,<br></div>Vithulan.<br></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On 8 March 2016 at 18:05, Juhász, Viktor <span dir="ltr"><<a href="mailto:viktor.juhasz@balabit.com" target="_blank">viktor.juhasz@balabit.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Vithulan,<div><br></div><div>Yes, this should be plugin (you can find the plugins in the modules directory in syslog-ng source).<br><div><br></div><div>Just a very briefly overview:<br><br>syslog-ng read messages from the sources, process them with filters, rewrite rules, parsers, etc, and finally send messages to the destinations.<br>We have a Kafka destination, it is implemented in java. The kafka source also should be implemented in java, but we our java source base is under construction, so till I can't give you an exact API, it is comming in the next 2-3 weeks.<br><br>The ovarall idea is the following.<br> - We should implement a Kafka consumer which uses the high level kafka group consuming API (<a href="https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example" target="_blank">https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example</a>)<br> - We have to process data read from the Kafka (create LogMessage from this. LogMessage is an internal struct in syslog-ng representing the in the software)<br> - After reload or restart syslog-ng have to be able to continue reading messages from the last read message.</div><div> - If there are more syslog-ng's reading the same Kafka input (these syslog-ng's have the same group name) avoid message loosing or message duplication as much as possible<br><br>Regards,</div><div>Viktor</div><div><br><br></div></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 7, 2016 at 6:43 PM, Vithulan MV <span dir="ltr"><<a href="mailto:vithulanmv.12@cse.mrt.ac.lk" target="_blank">vithulanmv.12@cse.mrt.ac.lk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p dir="ltr">Hi Viktor,</p>
<p dir="ltr">Oh I see! I've sent a mail to the mail list. Can you kindly give me an overall idea of project Kafka source please? (Architecture of syslog-ng/ current APIs) Is this is going to be an extension of syslog-ng? </p>
<p dir="ltr">Thank you.</p>
<p dir="ltr">Regards, <br>
Vithulan.</p><div><div>
<div style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hi Vithulan,<div><br></div><div>I've never faced with this problem, during compiling syslog-ng.<br>I suggest, that ask about your problem in the mail list of syslog-ng ;)</div><div><br></div><div>Best Regards,</div><div>Viktor<br><br></div></div><div class="gmail_extra"><br><div class="gmail_quote">On Mon, Mar 7, 2016 at 12:46 PM, Vithulan MV <span dir="ltr"><<a href="mailto:vithulanmv.12@cse.mrt.ac.lk" target="_blank">vithulanmv.12@cse.mrt.ac.lk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Hi,<br><br></div>I've been going through the git book that you referred. It was helpful. Can you please explain a bit about the project structure and deliverables of Kafka-source project? <br><br></div>I added required dependencies and tried to build syslog-ng from the source, there was an error saying, <br><i>${syslog-ng_HOME}/modules/afmongodb/mongo-c-driver/src/mongoc/mongoc-scram.c: In function '_mongoc_scram_sha1':</i> <b>storage size of 'digest_ctx' isn't known</b><i> (EVP_MD_CTX digest_ctx)</i><b><br></b></div>I confirmed OpenSSL [1] headers are included in the file.Can you please point out whats wrong? <br><br></div>In the meantime I'm going through the syslog-ng documentation [2]. Can you give me directives on what I can be doing while preparing my proposals?<br><div><br>[1] <a href="https://github.com/mongodb/mongo-c-driver/blob/7415d3468de0c500fd6e630f7b52be181cbb66b3/src/mongoc/mongoc-scram.c#L311" target="_blank">https://github.com/mongodb/mongo-c-driver/blob/7415d3468de0c500fd6e630f7b52be181cbb66b3/src/mongoc/mongoc-scram.c#L311</a><br>[2] <a href="https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/so-contents.html" target="_blank">https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-guides/en/syslog-ng-ose-guide-admin/html/so-contents.html</a><br><br></div><div>Thank you.<br></div><div>Regards,<br></div><div>Vithulan.<br></div><div><br></div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On 4 March 2016 at 18:56, Vithulan MV <span dir="ltr"><<a href="mailto:vithulanmv.12@cse.mrt.ac.lk" target="_blank">vithulanmv.12@cse.mrt.ac.lk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Hi,<br><br></div>Thanks for the reference, I will go through it and report you back on the progress. <br><br></div>Thanks,<br></div>Regards,<br></div>Vithulan. <br></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On 4 March 2016 at 18:45, Juhász, Viktor <span dir="ltr"><<a href="mailto:viktor.juhasz@balabit.com" target="_blank">viktor.juhasz@balabit.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr">Hello,<div><br></div><div>Sorry for the long response time. We are working on a wiki page, this page will describe all the required information. Till please read our documentation (getting started) <a href="https://www.gitbook.com/book/syslog-ng/getting-started/details" target="_blank">https://www.gitbook.com/book/syslog-ng/getting-started/details</a><br></div><div><br></div><div>Best Regards,</div><div>Viktor Juhasz</div></div><div><div><div class="gmail_extra"><br><div class="gmail_quote">On Fri, Mar 4, 2016 at 2:09 PM, Vithulan MV <span dir="ltr"><<a href="mailto:vithulanmv.12@cse.mrt.ac.lk" target="_blank">vithulanmv.12@cse.mrt.ac.lk</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div>Hello,<br><br></div>I sent the following mail to the syslog-ng mailing list, but didn't get any responses. So I thought it didn't reach to you. I'm highly motivated on contributing for syslog-ng in GSoC 2016. I also have prior experience in syslog-ng domain so I feel like this project is going to be very interesting. Please kindly guide me on how I should start? <br><br></div>Thank you.<br><br></div>Regards,<br></div>Vithulan.<div><div><br><div><div><div><div><br><div><div class="gmail_quote">---------- Forwarded message ----------<br>From: <b class="gmail_sendername">Vithulan MV</b> <span dir="ltr"><<a href="mailto:vithulanmv.12@cse.mrt.ac.lk" target="_blank">vithulanmv.12@cse.mrt.ac.lk</a>></span><br>Date: 3 March 2016 at 15:33<br>Subject: [GSoC] Project: Kafka source<br>To: <a href="mailto:syslog-ng@lists.balabit.hu" target="_blank">syslog-ng@lists.balabit.hu</a><br><br><br><div dir="ltr"><div><div><div>Hi all, <br><br>I'm currently a 3rd year undergraduate at University of Moratuwa, Sri
lanka. I have attached my Linkedin profile with this mail [1].
I'd like to contribute to a GSoC 2016 project with the open source
community of syslog-ng.<br><br></div>I have contributed in log analyzing project LogAnalyzer [2] and also worked with logstash, kafka. Therefore, I think I can contribute syslog-ng project kafka source as I'm much interested in that.<br><br></div>Please kindly make any suggestions on how I should proceed on this?<br></div><div><div><div><br>[1] <a href="https://lk.linkedin.com/in/vithulamv" target="_blank">https://lk.linkedin.com/in/vithulamv</a><br>[2] <a href="https://github.com/VIthulan/product-la" target="_blank">https://github.com/VIthulan/product-la</a><br><br></div><div>Thank you.<br></div><div>Regards,<br></div><div>Vithulan<span><font color="#888888"><br></font></span></div><span><font color="#888888"><div><div><br>-- <br><div><div dir="ltr"><div style="font-size:small"><span style="font-family:monospace,monospace">Vithulan MV.</span><br></div><div style="font-size:small"><font face="monospace, monospace"><br></font></div><div style="font-size:small"><font face="monospace, monospace">Undergraduate, </font></div><div style="font-size:small"><font face="monospace, monospace">Department of Computer Science & Engineering,</font></div><div style="font-size:small"><font face="monospace, monospace">University of Moratuwa,</font></div><div style="font-size:small"><font face="monospace, monospace">Sri lanka.</font></div></div></div>
</div></div></font></span></div></div></div>
</div><br><br clear="all"><br>-- <br><div><div dir="ltr"><div style="font-size:small"><span style="font-family:monospace,monospace">Vithulan MV.</span><br></div><div style="font-size:small"><font face="monospace, monospace"><br></font></div><div style="font-size:small"><font face="monospace, monospace">Undergraduate, </font></div><div style="font-size:small"><font face="monospace, monospace">Department of Computer Science & Engineering,</font></div><div style="font-size:small"><font face="monospace, monospace">University of Moratuwa,</font></div><div style="font-size:small"><font face="monospace, monospace">Sri lanka.</font></div></div></div>
</div></div></div></div></div></div></div></div>
</blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div><div dir="ltr"><div style="font-size:small"><span style="font-family:monospace,monospace">Vithulan MV.</span><br></div><div style="font-size:small"><font face="monospace, monospace"><br></font></div><div style="font-size:small"><font face="monospace, monospace">Undergraduate, </font></div><div style="font-size:small"><font face="monospace, monospace">Department of Computer Science & Engineering,</font></div><div style="font-size:small"><font face="monospace, monospace">University of Moratuwa,</font></div><div style="font-size:small"><font face="monospace, monospace">Sri lanka.</font></div></div></div>
</div>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div><div dir="ltr"><div style="font-size:small"><span style="font-family:monospace,monospace">Vithulan MV.</span><br></div><div style="font-size:small"><font face="monospace, monospace"><br></font></div><div style="font-size:small"><font face="monospace, monospace">Undergraduate, </font></div><div style="font-size:small"><font face="monospace, monospace">Department of Computer Science & Engineering,</font></div><div style="font-size:small"><font face="monospace, monospace">University of Moratuwa,</font></div><div style="font-size:small"><font face="monospace, monospace">Sri lanka.</font></div></div></div>
</div>
</div></div></blockquote></div><br></div>
</div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div><div dir="ltr"><div style="font-size:small"><span style="font-family:monospace,monospace">Vithulan MV.</span><br></div><div style="font-size:small"><font face="monospace, monospace"><br></font></div><div style="font-size:small"><font face="monospace, monospace">Undergraduate, </font></div><div style="font-size:small"><font face="monospace, monospace">Department of Computer Science & Engineering,</font></div><div style="font-size:small"><font face="monospace, monospace">University of Moratuwa,</font></div><div style="font-size:small"><font face="monospace, monospace">Sri lanka.</font></div></div></div>
</div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div><div dir="ltr"><div style="font-size:small"><span style="font-family:monospace,monospace">Vithulan MV.</span><br></div><div style="font-size:small"><font face="monospace, monospace"><br></font></div><div style="font-size:small"><font face="monospace, monospace">Undergraduate, </font></div><div style="font-size:small"><font face="monospace, monospace">Department of Computer Science & Engineering,</font></div><div style="font-size:small"><font face="monospace, monospace">University of Moratuwa,</font></div><div style="font-size:small"><font face="monospace, monospace">Sri lanka.</font></div></div></div>
</div>
</div></div></blockquote></div><br></div>
</div></div></blockquote></div><br><br clear="all"><br>-- <br><div><div dir="ltr"><div style="font-size:small"><span style="font-family:monospace,monospace">Vithulan MV.</span><br></div><div style="font-size:small"><font face="monospace, monospace"><br></font></div><div style="font-size:small"><font face="monospace, monospace">Undergraduate, </font></div><div style="font-size:small"><font face="monospace, monospace">Department of Computer Science & Engineering,</font></div><div style="font-size:small"><font face="monospace, monospace">University of Moratuwa,</font></div><div style="font-size:small"><font face="monospace, monospace">Sri lanka.</font></div></div></div>
</div>
</blockquote></div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div><div dir="ltr"><div style="font-size:small"><span style="font-family:monospace,monospace">Vithulan MV.</span><br></div><div style="font-size:small"><font face="monospace, monospace"><br></font></div><div style="font-size:small"><font face="monospace, monospace">Undergraduate, </font></div><div style="font-size:small"><font face="monospace, monospace">Department of Computer Science & Engineering,</font></div><div style="font-size:small"><font face="monospace, monospace">University of Moratuwa,</font></div><div style="font-size:small"><font face="monospace, monospace">Sri lanka.</font></div></div></div>
</div>
</div></div></blockquote></div><br><br clear="all"><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr"><div style="font-size:small"><span style="font-family:monospace,monospace">Vithulan MV.</span><br></div><div style="font-size:small"><font face="monospace, monospace"><br></font></div><div style="font-size:small"><font face="monospace, monospace">Undergraduate, </font></div><div style="font-size:small"><font face="monospace, monospace">Department of Computer Science & Engineering,</font></div><div style="font-size:small"><font face="monospace, monospace">University of Moratuwa,</font></div><div style="font-size:small"><font face="monospace, monospace">Sri lanka.</font></div></div></div>
</div>