March 2016 - syslog-ng - lists.balabit.hu

syslog-ng-3.7.3
by Budai, László 22 Mar '16

22 Mar '16

3.7.3 Changes compared to 3.7.2: Improvements - Updated Python package requirements. - Can now compile without MongoDB. - Added eventlog to the list of required pkg-config packages. - Basic FreeBSD and HP-UX support of syslog debug bundle generator by improving POSIX shell compatibility. - Keep the program destination open between configuration reloads. - system-source now uses keep-timestamp(no) for Linux kernel log. The time source used by /dev/kmsg is not updated after system SUSPEND/RESUME. Fixes - Fix a SIGSEGV when a Redis command returns an error. - Resolve deadlock in logwriter triggered by suppress() - Mitigate possible deadlock in patterndb - Fixed global inheritance of pass-unix-credentials() and create-dirs(). - Certain compilers complained about an undefined symbol when setting keep-alive(yes). - For certain use cases, afsocket would not handle procfs read errors due to an integer underflow. - Enhanced Java version check and the handling of SyslogNgInternalLogger (used by Kafka), the FATAL loglevel and getLocationInformation(). - When a big amount of kernel log was produced in a very short time, the syslog-ng process sometimes entered into a spin and stop processing messages. Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: András Mitzki, Avleen Vig, Balázs Scheidler, Ben Kibbey, Christian Herzig, David Schweikert, Douglas Carmichael, Dezso Endre Molnar, Fabien Wernli, Gergely Czuczy, Gergely Nagy, Gergo Nagy, Hanno Böck, Herzig, Christian, Laszlo Budai, László Várady, MÓZES Ádám István, PÁSZTOR György, Péter Czanik, Robert Fekete, Saurabh Shukla, Tamás Nagy, Tibor Benke, Viktor Juhász, Vincent Bernat, Wang Long, Zdenek Styblik, Zoltán FRIED, Zoltán Pallagi — You are receiving this because you are subscribed to this thread. View it on GitHub <https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.7.3>

1 0

[GSoC] Interested in Python support for Correlation Library
by Akshay Aurora 22 Mar '16

22 Mar '16

Hi, I'm Akshay, computer science student from India. I have decent experience with Python and system programming. Being a Mozillian, I have always been a big fan of Rust and Servo project. I have previously dabbled with Rust – and think of it as next system programming language. I hope to improve my Rust skills as part of this project and write production quality code. I have also been Google Summer of Code student at Inclusive Design Institute in 2013 and Mozilla in 2014. Few questions and ideas related to the project - - While the reason to switch away from handlebars makes sense - Why do we want to using Python as templating language? What are some of the use cases, where Python could be beneficial over any other templating language? - Are there any existing projects using Python as templating language? How is this going to be different than passing the context data to a Python script? - For the SQL-like syntax mentioned in project description - Can we design it as Python library using Rust via CFFI [0]. I have filled the student application form. I'm already trying to understand the correlation library and playing around with existing tests. How can I get more involved and better understand the requirements of the project? [0] - https://github.com/tremolo/rust_python_integration -- - Akshay Aurora ᐧ

2 3

syslog-ng 3.7.2
by Budai, László 22 Mar '16

22 Mar '16

3.7.2 This is the first maintenance release for the 3.7.x series. Changes compared to 3.7.1: Improvements - Added mbox() source. This source can be used to fetch emails from local mbox files: source { mbox("/var/spool/mail/root"); }; This will fetch root emails and parse them into a multiline $MSG. Original implementation by Fabien Wernli, I only converted it into an SCL. - It is possible to append dynamically options into SCL blocks from now. - concurrent_request option added to ElasticSearch with default value 1. - In elasticsearch destinaton, message_template() argument renamed to template(). - SCL added to every Java module (ElasticSearch, Kafka, HDFS). - Linux Audit Parser added for parsing key-value pairs produced by the Linux Audit subsystem. - HTTP destination is now able to receive HTTP method as an option. All the supported methods are available (POST, PUT, HEAD, OPTIONS, DELETE, TRACE, GET). Fixes - In some circumstances syslog-ng mod-journal re-read every already processed messages. - When syslog-ng got a reload and the reload process done within 1 second then mafter the reload, syslog-ng stop generating mark-messages. - When initialization of a network destination in syslog-ng failed (eg. due to DNS resolution failure) we didn't create a queue which caused message loss. - syslog-ng segfaulted on TLS errors when wrong certs was provided (eg.: CA cert with the cert-file directive instead of the server cert). - Fixed a continuous spinning case in the file driver, when the destination file is a device (e.g. /dev/stdout). - A memory leak in around template functions in grammar fixed. - Fixed Python3 support. - Fixed Python GIL issue in python destination. - From now, instead of skipping doc/ alltogether when ENABLE_MANPAGES is not set, only skip the actual man pages, but handle the rest properly. - Allow overriding the python setup.py options. When installing the python modules, allow overriding the options. This is useful for distributions that want to pass extra options. For example, on Debian, we want --install-layout="deb" instead of the --prefix and --root options. With this change, the previous behaviour remains the default, but one can supply PYSETUP_OPTIONS on the make command-line to override it. - The systemd service file read /etc/default/syslog-ng and /etc/sysconfig/syslog-ng, but didn't do anything with their contents. $SYSLOGNG_OPTS added to ExecStart, so that the EnvironmentFiles have an effect (at least on Debian). - Java support checking fixed (not only jdk is required but also gradle). - Memory leak around ping() in Redis fixed. - A crash in pdbtool fixed around r_parser_email(). - Removed cygwin fdlimit statement. Make the default for RLIMIT_NOFILE equal to the current system limits. --fd-limit can still override this, but the default will be configured based on existing system limits. - Fixed BSD year inference. Fixed logic and made clearer the inference of year from bsd-style rfc3164 syslog-messages, which do not include a year. - Handle correctly the epoch 0 timestamp. (Previously, syslog-ng cached the zero timestamp and treated 1970 as it was 1900.) Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Adam Arsenault, Adam Istvan Mozes, Andras Mitzki, Avleen Vig, Balazs Scheidler, Fabien Wernli, Gergely Czuczy, Gergely Nagy, Gergo Nagy, Laszlo Budai, Peter Czanik, Robert Fekete, Saurabh Shukla, Tamas Nagy, Tibor Benke, Viktor Juhasz, Vincent Bernat, Wang Long, Zdenek Styblik, Zoltan Pallagi. — View it on GitHub <https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.7.2>.

4 3

error='self signed certificate in certificate chain'
by Girish Kumar 17 Mar '16

17 Mar '16

Hi All, I am getting following error while starting syslog-ng with tls option. Could you please help me on this Mar 12 18:14:24 (none) syslog-ng[6136]: Syslog connection established; fd='5', server='AF_INET(10.135.83.103:6514)', local='AF_INET(0.0.0.0:0)' Mar 12 18:14:24 (none) syslog-ng[6136]: Certificate validation failed; subject='emailAddress=giri(a)gmail.com, CN=girish kumar, OU=esd, O=enterprise, L=BAN, ST=KA, C=IN', issuer='emailAddress=giri(a)gmail.com, CN=girish kumar, OU=esd, O=enterprise, L=BAN, ST=KA, C=IN', error='self signed certificate in certificate chain', depth='1' Mar 12 18:14:24 (none) syslog-ng[6136]: SSL error while writing stream; tls_error='SSL routines:ssl3_get_server_certificate:certificate verify failed' Mar 12 18:14:24 (none) syslog-ng[6136]: I/O error occurred while writing; fd='5', error='Broken pipe (32)' Mar 12 18:14:24 (none) syslog-ng[6136]: Syslog connection broken; fd='5', server='AF_INET(10.135.83.103:6514)', time_reopen='60' //server conf source d_source { #syslog(ip("mysyslog.server.com") port(6514) syslog(ip("10.135.83.103") port(6514) transport("tls") tls( key_file("/etc/cert.d/mySerPrivate.key") cert_file("/etc/cert.d/mySerCert.pem") ca_dir("/etc/ca.d") ssl-options(no-sslv2, no-sslv3, no-tlsv1, no-tlsv11) ) ); }; //Client conf destination d_destination { #syslog("mysyslog.server.com" port(6514) syslog("10.135.83.103" port(6514) transport("tls") tls( ca_dir("/etc/ca.d") key_file("/etc/cert.d/myCliPrivate.key") cert_file("/etc/cert.d/myCliCert.pem") ssl-options(no-sslv2, no-sslv3, no-tlsv1, no-tlsv11) ) ); }; Regard, Girish

2 5

[GSOC]Some questions about the project:WebSocket for syslog-ng
by Yilin Li 17 Mar '16

17 Mar '16

Hi, I got some questions about GSOC Project: WebSocket for syslog-ng 1) WebSocket support two-way communication. However, I found only one-way communication will be used in the project Idea. If this destination will be used for alerting, what are the advantages of using Websocket? Is it mainly for supporting another new protocol or better performance of WebSocket ? 2) It says we need a sample webserver to receive and handle messages coming from syslog-ng. Can this sample webserver written in python ? Does "handle" just mean recieve messages and display them to prove the destination is working properly? Need the webserver do anything else such as writing them to files or sending emails for altering? Thanks :) -- Thanks, Yiln Li -- Institute of Software Chinese Academy of Sciences

3 5

[GSoC]
by Juhász, Viktor 16 Mar '16

16 Mar '16

Dear GSoC Student Applicant, We would like to thank you for filling the Student Application Form. To finish your application process, do not forget to write your proposal Best Regards, Viktor Juhasz The syslog-ng project

1 0

Re: [syslog-ng] syslog-ng GSoC 2016
by Laszlo Budai 11 Mar '16

11 Mar '16

Hi, please read the GSoC docs provided by Google: https://developers.google.com/open-source/gsoc/rules They say that : Each Student may submit up to five (5) Project Proposals; however, only one (1) Project Proposal may be accepted per Student. So the short answer is yes, you can :) L. Sent from my HTC ----- Reply message ----- From: "Yilin Li" <liyilin1214(a)gmail.com> To: "Syslog-ng users' and developers' mailing list" <syslog-ng(a)lists.balabit.hu> Subject: [syslog-ng] syslog-ng GSoC 2016 Date: Thu, Mar 10, 2016 17:06 Hi, Really grateful for your guidance. I've got some questions. If I'm interested in more than one projects. Can I submit more than one proposals? Thanks! 2016-03-10 23:24 GMT+08:00 Budai, László <laszlo.budai(a)balabit.com>: > Hi, > > the syslog-ng team invites you to join GSoC with us : > http://youtu.be/hCLOGU8OooM . > > If you are interested in GSoC, visit our homepage where we share some > useful information with you: https://syslog-ng.org/gsoc-2016/ . > > > regards, > Laszlo Budai > > > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > > > -- Thanks, Yiln Li -- Institute of Software Chinese Academy of Sciences ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

2 1

syslog-ng GSoC 2016
by Budai, László 10 Mar '16

10 Mar '16

Hi, the syslog-ng team invites you to join GSoC with us : http://youtu.be/hCLOGU8OooM . If you are interested in GSoC, visit our homepage where we share some useful information with you: https://syslog-ng.org/gsoc-2016/ . regards, Laszlo Budai

2 1

[GSoC] Project: Kafka source
by md mehrab 10 Mar '16

10 Mar '16

Hi all, I am currently 4th year undergraduate student at Faculty of Engineering, Jamia Millia Islamia, New Delhi, India. I would like to contribute to a GSoC 2016 project with open source community of syslog-ng. In last summer vacation i worked (as intern )for a company where i was the part of data aggregator project (one of the core member of this project ). Main idea of this project is to build scalable system so that it can handle millions of request per minute. For this we used kafka messaging system. Producer simply produce message to particular topic and consumer get message from that topic and feed to async task (celery worker). I think I can contribute syslog-ng project kafka source. @Viktor Juhasz <https://github.com/juhaszviktor> (Mentor) kindly give me suggestions. -- Md Mehrab Alam https://www.linkedin.com/in/iammehrabalam https://github.com/iammehrabalam

2 3

Insider 2016-03: GSoC; disk buffer; Rust; blogs; DevConf.cz
by Czanik, Péter 10 Mar '16

10 Mar '16

Dear syslog-ng users, This is the 48th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Google Summer of Code (GSoC) ---------------------------- The syslog-ng project will participate GSoC this year again. We published our application form: https://syslog-ng.org/gsoc-2016-applicant-form-syslog-ng-project/ As you can see, we plan to publish a GsoC release once the coding period is over. Topics cover many different aspects of syslog-ng, from release engineering through database connections to message correlation. Disk-based buffering -------------------- Disk-based buffering is now merged to syslog-ng git master and will be part of syslog-ng 3.8. It can be used for storing messages on the local hard disk if the central log server or the network connection to the server becomes unavailable. The syslog-ng application automatically sends the stored messages to the server when the connection is reestablished. For details and information how to test it now read https://syslog-ng.org/493-2/ Rust: Filling the parse() method -------------------------------- Tibor Benke has posted an article about writing a parser that uses regular expressions in Rust. Read more at https://syslog-ng.org/filling-the-parse-method/ Parsers in syslog-ng: making sense of Zorp firewall logs -------------------------------------------------------- Figuring out how to parse your firewall logs is not always easy. This blogpost shows you some useful log-parsing techniques. For the examples, I use the logs of the Zorp proxy firewall. https://czanik.blogs.balabit.com/2016/03/making-sense-of-zorp-firewall-logs… Community driven blogging ------------------------- >From time to time, we publish blog posts about syslog-ng and we hope you find them interesting and useful. However, we would like to become better at this: we want to give you more and better content, more regularly. Also, we would like to be transparent about it, and give you the possibility to get involved. Read how you can participate: https://syslog-ng.org/525-2/ DevConf.cz ---------- This year it was even more difficult to decide how to spend my time at DevConf, the annual Fedora, Red Hat, JBoss developers’ conference in Brno. As a community manager for syslog-ng, I have tried to focus on community-related presentations and on technologies related to syslog-ng: containers, security and packaging. https://czanik.blogs.balabit.com/2016/02/devconf-2016-community-and-contain… NEW RELEASES syslog-ng PE 5 F6: https://istvanszabo.blogs.balabit.com/2016/03/syslog-ng-premium-edition-5f6… Your feedback and news tips about the next issue is welcome at documentation(a)balabit.com . To read this newsletter on-line, visit: http://insider.blogs.balabit.com/ Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0