- syslog-ng - lists.balabit.hu

unofficial syslog-ng 3.12 RPM packages
by Czanik, Péter 25 Sep '17

25 Sep '17

Hi, Syslog-ng 3.12 was released this morning. I have now my unofficial RPM packages available for openSUSE / SLES 12 / EPEL 7 (RHEL, CentOS, etc.). Fedora is missing from the list due to https://github.com/balabit/syslog-ng/issues/1669 You can read my updated installation instructions at https://www.balabit.com/blog/installing-latest-syslog-ng-on-rhel-and-other-… Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

1 0

OSE 3.12
by Laszlo Budai 25 Sep '17

25 Sep '17

3.12.1 Features * HDFS: support macro in filename (#1638<https://github.com/balabit/syslog-ng/pull/1638>) * HDFS: add append support (#1675<https://github.com/balabit/syslog-ng/pull/1675>) * Java: allow to use sequence numbers in templates (#1628<https://github.com/balabit/syslog-ng/pull/1628>) * TLS improvements (#1603<https://github.com/balabit/syslog-ng/pull/1603>, #1636<https://github.com/balabit/syslog-ng/pull/1636>) * Add PKCS 12 support with the new pkcs12-file() TLS option * startup time ssl-options() and peer-verify() check * startup time key_file, cert_file, ca_dir, crl_dir and cipher_suite check * ECDH cipher support (OpenSSL 1.0.1, 1.0.2, 1.1.0) with the ecdh-curve-list() option (only available >= 1.0.2) * for < 1.0.2, a hard-coded curve is used * for >= 1.0.2, automatic curve selection is used (the ecdh-curve-list() option can restrict this list) * DH cipher support with the dhparam-file() option * if the option is not specified, fallback RFC 3526 parameters are used * minor fixes * stdin() source driver (#1605<https://github.com/balabit/syslog-ng/pull/1605>) * Implement read_old_records option for systemd-journal source (#1642<https://github.com/balabit/syslog-ng/pull/1642>) * Add tags-parser: a new module to parse $TAGS values (#1658<https://github.com/balabit/syslog-ng/pull/1658>) * Add a Windows eventlog parser scl module (#1572<https://github.com/balabit/syslog-ng/pull/1572>) * Add XML parser module (#1659<https://github.com/balabit/syslog-ng/pull/1659>, #1684<https://github.com/balabit/syslog-ng/pull/1684>) Bugfixes * Fix cannot parse ipv6 into hostname (#1617<https://github.com/balabit/syslog-ng/pull/1617>) * Speedup add-contextual-data by making ordering optional (#1645<https://github.com/balabit/syslog-ng/pull/1645>) * Fix monitor-method() option not working for wildcard-file() source (#1651<https://github.com/balabit/syslog-ng/pull/1651>) * Sanitize SDATA keys in syslog-protocol messages to avoid generating non-valid messages (#1650<https://github.com/balabit/syslog-ng/pull/1650>, #1654<https://github.com/balabit/syslog-ng/pull/1654>) * Fix memory leaks reported using Valgrind (#1649<https://github.com/balabit/syslog-ng/pull/1649>) * Fix memory leak related to cloning pipes and reload (#1647<https://github.com/balabit/syslog-ng/pull/1647>) * Fix getent protocol number returns incorrect value (#1665<https://github.com/balabit/syslog-ng/pull/1665>) * Fix elasticsearch2 destination flush mechanism (#1668<https://github.com/balabit/syslog-ng/pull/1668>) * Fix file destination related memory leak (#1685<https://github.com/balabit/syslog-ng/pull/1685>) * Fix a possible memory leak around affile destination (#1685<https://github.com/balabit/syslog-ng/pull/1685>) Other changes * Improve syslog-ng debun functionality (#1633<https://github.com/balabit/syslog-ng/pull/1633>, #1641<https://github.com/balabit/syslog-ng/pull/1641>, #1663<https://github.com/balabit/syslog-ng/pull/1663>) * Java: allow to set JVM options form global syslog-ng options (#1639<https://github.com/balabit/syslog-ng/pull/1639>) * Do steps towards Python 3 support: * Fix string compatibility for Python 3 (#1632<https://github.com/balabit/syslog-ng/pull/1632>) * Improve Python version auto detection (#1660<https://github.com/balabit/syslog-ng/pull/1660>) * HTTP destination: display verbose logs on debug level (#1526<https://github.com/balabit/syslog-ng/pull/1526>) * Improvements for Solaris packing (#1664<https://github.com/balabit/syslog-ng/pull/1664>) Notes to the Developers * Update internal RabbitMQ (#1662<https://github.com/balabit/syslog-ng/pull/1662>) * Update internal ivykis to v0.42 (#1566<https://github.com/balabit/syslog-ng/pull/1566>) * Fix Travis and test related issues (#1566<https://github.com/balabit/syslog-ng/pull/1566>, #1644<https://github.com/balabit/syslog-ng/pull/1644>, #1674<https://github.com/balabit/syslog-ng/pull/1674>) * Update docker images (#1637<https://github.com/balabit/syslog-ng/pull/1637>) * Fix some clang compile errors (#1662<https://github.com/balabit/syslog-ng/pull/1662>) Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Andras Mitzki, Antal Nemes, Attila Szalay, Balazs Scheidler, Gabor Nagy, Gergely Orosz, Janos Szigetvari, Laszlo Budai, Laszlo Varady, Mate Farkas, Marton Suranyi, Peter Kokai, Szilard Pfeiffer, Tamas Nagy, Zoltan Pallagi. https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.12.1 [https://avatars1.githubusercontent.com/u/986981?v=4&s=400]<https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.12.1> balabit/syslog-ng<https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.12.1> syslog-ng is an enhanced log daemon, supporting a wide range of input and output methods: syslog, unstructured text, queueing, SQL & NoSQL. github.com

1 0

upgrading syslog-ng on Debian/Ubuntu packages from unofficial OBS repository
by Laszlo Budai 22 Sep '17

22 Sep '17

Hi, if you got the following error: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY xxxxxx just update the APT key: curl -4 http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUb… | sudo apt-key add - (if you are using other version of Debian/Ubuntu, set the correct version) and then apt-get update && apt-get upgrade regards, Laszlo Budai

1 0

unofficial OSE 3.12 packages for Debian/Ubuntu
by Laszlo Budai 22 Sep '17

22 Sep '17

Hi, I'm starting to work on upgrading my packages in OBS. Please test the packages before upgrading your servers. If you find any issues, just drop me a mail and I'll try to fix them. regards, Laszlo Budai

1 0

Insider 2017-09: 3.12; survey; geoip2; docker; AWS;
by Czanik, Péter 21 Sep '17

21 Sep '17

Dear syslog-ng users, This is the 61th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS syslog-ng 3.12 is coming ------------------------ The latest version of syslog-ng, 3.12 is expected to arrive soon. The most important new feature is the XML parser, which creates name-value pairs from XML-based log messages. You will also be able to configure JVM options for Java destinations, use PKCS-12, and utilize macros in HDFS file names. There are many more smaller features and bug fixes. For a complete list, check the git commits: https://github.com/balabit/syslog-ng/commits/master Test RPM packages from git head: https://www.balabit.com/blog/rpm-packages-from-syslog-ng-git-head/ New syslog-ng website --------------------- We will introduce a brand new website for all that is syslog-ng, and with that will come an official syslog-ng OSE repository with binaries for some of the most popular Linux distributions. We expect to launch syslog-ng.com in November, moving content for both OSE and its commercial versions to this new site so that it can serve as a single hub for all syslog-ng users. Learn more about it and help our work by answering a brief survey: https://www.balabit.com/blog/new-website-and-official-repo-coming-up-tell-u… Collecting logs from containers ------------------------------- There are many software products that log to files or pipes instead of their stdout, the place where Docker expects them. Fortunately by using Docker volumes, you can share data among containers, and syslog-ng can collect these logs as well. Learn how: https://www.balabit.com/blog/collecting-logs-containers-using-docker-volume… Heat maps using the new geoip2 parser ------------------------------------- The new geoip2 parser of syslog-ng 3.11 is not only faster than its predecessor, but can also provide a lot more detailed geographical information about IP addresses. Next to the usual country name and longitude/latitude information, it also provides the continent, time zone, postal code and even county name. Some of these are available in multiple languages. Learn how you can utilize all this information by parsing logs from iptables using syslog-ng, storing them to Elasticsearch, and displaying the results in Kibana! https://www.balabit.com/blog/creating-heat-maps-using-new-syslog-ng-geoip2-… Learn how you can impress your C-level leader(s) even more by animating these heat maps: https://www.balabit.com/blog/creating-time-lapse-videos-log-messages-using-… Installing syslog-ng on AWS Linux AMI ------------------------------------- You do not have to live without your favorite syslog implementation even in Amazon Web Services (AWS) Linux AMI. This Linux distribution is based on Red Hat Enterprise Linux version 6 and it takes minimal extra work to install syslog-ng on it. https://www.balabit.com/blog/installing-syslog-ng-aws-linux-ami/ EVENTS Virtualization day in Budapest, November 7th: https://vday.hu/2017/ Your feedback and news, or tips about the next issue are welcome at documentation(a)balabit.com. To read this newsletter online, visit: https://syslog-ng.org/ Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

1 0

New website and official repo coming up, tell us what you think!
by Czanik, Péter 19 Sep '17

19 Sep '17

Hi, We at Balabit are and have always been both very excited and humbled by the community that formed around syslog-ng over the past two decades. Many of you have been a part of this long journey and helped syslog-ng to get to where it is today, and we thank you for that. Throughout the years, we provided a best-in-class documentation not just for our commercial offerings but for syslog-ng Open Source Edition as well. However, for the past half decade official binaries existed only for syslog-ng Premium Edition. Soon, this will change. We will introduce a brand new website for all that is syslog-ng, and with that will come an official syslog-ng OSE repository with binaries for some of the most popular Linux distributions. We expect to launch syslog-ng.com in November, moving content for both OSE and its commercial versions to the site to become a single hub for all syslog-ng users. The official syslog-ng OSE repository will be just a part (although a major one) of a comprehensive overhaul of our OSE related content providing platform. We will provide regular technical webinars and trainings for syslog-ng OSE users, more blogs on practical tips&tricks, common use cases, etc. We plan to channel all syslog-ng related discussion into a forum on the new site, although this will take some more time. And we already started putting in more resources into the development of the Open Source Edition. This is a major commitment on our side to the community, and we know that it is the right move. However, we haven't quite decided on the implementation yet, and we are looking for your feedback on this. This is a serious business decision and ultimately all stakeholders (we as a company and you as a member of the community) have to benefit from this move, otherwise there is no reason to do it. We are listing all these so you can see the bigger picture before we present you the question: Would you register on the upcoming syslog-ng website to get access to the official repository and the aforementioned content? What would you gain if you registered? - Access to a repository containing the latest syslog-ng packages (technically, you would receive a key that you could include in FIXME) - Alerts, announcements and news about the syslog-ng releases - Regular news about syslog-ng related content to help you better solve problems with syslog-ng (blog posts, tutorials, interesting use cases about problems that others solved with syslog-ng, and so on) What would we gain if you registered? - First and foremost, we would get information immensely useful for us to optimize our development resources. How many of you use the repositories, which versions of syslog-ng you use, which platforms are the most important, which ones you do not use anymore, and so on. Unfortunately, we cannot get such data if you use external repositories (like the current unofficial ones), or if you install syslog-ng from your distribution. - Information about in which countries are hubs of syslog-ng users, for example, where could we organize meetups most successfully, or participate in local conferences to strengthen the community. - The ability to send you information about syslog-ng, both to help you learn about the possibilities of syslog-ng, and also to let you know about our commercial offerings. We know that some of you won't be interested in anything commercial, and that's fine. But to keep syslog-ng alive, we also need the support of those who do. We believe that registering on the new website and receiving the occasional email from us is a small favor to ask in return of the convenience of having access to a official repository with up-to-date packages, and we will strive to send you emails with useful content. Please let us know what you think using this short Google Form. <https://goo.gl/forms/TmTJY8CkRAzUwtEA2> Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

3 6

CentOS cant start syslog-ng via systemd but runs fine manually
by Andrew 15 Sep '17

15 Sep '17

I recently updated my CentOS server so there were new versions of Java and syslog-ng and I have been having issues ever since. If I run it from systemd then it fails to load, yet if I run it manually it works fine and pushes the logs into elasticsearch. When trying to run from systemd it complains about Java plugin not being found. systemctl start syslog-ng Error parsing destination, destination plugin java not found in destination block elasticsearch2 at line 2, column 3 If I run it manually by typing syslog-ng then it works just fine.

4 6

3.11 Multiple elastic destinations.
by Scot 14 Sep '17

14 Sep '17

Has anyone had success/failure using multiple ES destinations in syslog-ng. I am want to direct traffic to different indexes based on syslog-ng filters destination d_es { elasticsearch2( client-mode("http") index("syslog-ng_${YEAR}.${MONTH}.${DAY}") type("syslog") # Description: The type of the index. For example, type("test") template("$(format-json --scope rfc3164 --scope nv-pairs --exclude R_DATE --key ISODATE)\n") cluster-url("http://elasticsearch:9200/") concurrent-requests("5") # Number of concurrrent batches flush_limit("5000") # The number of messages in a single batch skip-cluster-health-check("yes") cluster("clustername") client_lib_dir("/usr/share/elasticsearch/lib") ); }; destination d_es_network { elasticsearch2( client-mode("http") index("network_${YEAR}.${MONTH}.${DAY}") type("syslog") # Description: The type of the index. For example, type("test") template("$(format-json --scope rfc3164 --scope nv-pairs --exclude R_DATE --key ISODATE)\n") cluster-url("http://elasticsearch:9200/") concurrent-requests("5") # Number of concurrrent batches flush_limit("5000") # The number of messages in a single batch skip-cluster-health-check("yes") cluster("clustername") client_lib_dir("/usr/share/elasticsearch/lib") ); };

2 1

Re: [syslog-ng] Syslog-ng on IBM System i
by Balazs Scheidler 13 Sep '17

13 Sep '17

This was a commercial offering from balabit which has been discontinued. The balabit sales team can possibly refer to an alternative, and i can connect you if you want. Just email me in private. On Sep 13, 2017 12:09 AM, "Lucas Ihara Alves" <lucas.alves(a)trtsp.jus.br> wrote: Dear Sir or Madam, I would like to know if Syslog-ng Open Source has a compiled version for IBM System i. If so, please, I would be grateful if anyone could send me the link to download. -- Yours Truly, ---- Lucas Alves ____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

1 0

Syslog-ng on IBM System i
by Lucas Ihara Alves 13 Sep '17

13 Sep '17

Dear Sir or Madam, I would like to know if Syslog-ng Open Source has a compiled version for IBM System i. If so, please, I would be grateful if anyone could send me the link to download. -- Yours Truly, ---- Lucas Alves

1 0