[syslog-ng] Insider 2019-11: logging to Elasticsearch; PE 6 to 7 upgrade; Elastic 7; in-list(); off-line deb; Splunk conf;

[Peter Czanik (pczanik)]

Dear syslog-ng users,
This is the 76th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news.

NEWS

Logging to Elasticsearch made simple with syslog-ng
---------------------------------------------------
Elasticsearch is gaining momentum as the ultimate destination for log messages. You can store arbitrary name-value pairs coming from structured logging or message parsing and you can use Kibana as a search and visualization interface. You can use syslog-ng as an all in one solution (system and application logs, message parsing, filtering and queuing) to collect and forward log messages to Elasticsearch:
https://www.syslog-ng.com/community/b/blog/posts/logging-to-elasticsearch-made-simple-with-syslog-ng

Upgrading syslog-ng PE from version 6 to 7
------------------------------------------
Learn the major steps necessary to upgrade your system from syslog-ng Premium Edition version 6 to 7. As you will see, it is no more difficult than any other major software version upgrade, and after the upgrade you can start using all the new and useful features that are available in version 7.
https://www.syslog-ng.com/community/b/blog/posts/upgrading-syslog-ng-pe-from-version-6-to-7

syslog-ng and Elasticsearch 7: getting started on RHEL/CentOS
-------------------------------------------------------------
Version 7 of the Elastic Stack, packed with new features and improved performance, has now been available for some time. Elasticsearch is not the only one to have come up with a major new version recently: starting with version 3.21, syslog-ng features a new Elasticsearch destination driver (based on the http() destination) that does not require Java. In most cases, it is more resource friendly than the Java-based driver and it is definitely easier to configure. It also has the benefit that unlike the Java-based driver, it can be included in Linux distributions. This is a quick how-to guide to get you started with syslog-ng and Elasticsearch 7 on RHEL/CentOS 7.
https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-and-elasticsearch-7-getting-started-on-rhel-centos

Handling lists in syslog-ng: the in-list() filter
-------------------------------------------------
Recently, a number of quite complex configurations came up while syslog-ng users were asking for advice. Some of these configurations were even pushing the limits of syslog-ng (regarding the maximum number of configuration objects). As it turned out, these configurations could be significantly simplified using the in-list() filter, one of syslog-ng’s lesser known features.
https://www.syslog-ng.com/community/b/blog/posts/handling-lists-in-syslog-ng-the-in-list-filter

Offline syslog-ng DEB package installer
---------------------------------------
“How can I install the unofficial syslog-ng packages on a machine without Internet access?” This question has been raised several times recently. As it entails more than simply downloading the repository containing the packages, syslog-ng lead developer Laszlo Budai created a script that solves the problem for Debian and Ubuntu users. The script downloads syslog-ng along with its dependencies using a container, and produces an archive containing all DEB packages necessary to install syslog-ng as well as a simple script responsible for the installation. Using a container in this case might seem like over complication, but it is still the easiest way to ensure that all dependencies are included in the archive.
https://www.syslog-ng.com/community/b/blog/posts/offline-syslog-ng-deb-package-installer

syslog-ng at Splunk .conf 2019
------------------------------
The syslog-ng team participated at Splunk .conf 2019 and came home with positive experiences. They met enthusiastic users, who use syslog-ng in front of Splunk as log management layer. Many of them already using the HTTP event collector and use syslog-ng to load-balance log messages among collectors. They also demonstrated how syslog-ng can collect over one million log messages a second over UDP.
https://twitter.com/OneIdentity/status/1187852500222693377

WEBINARS

* You can achieve 1m+ UDP messages per second ingestion challenge: https://www.syslog-ng.com/webcast-ondemand/you-can-achieve-1m-udp-messages-per-second-ingestion-challenge8141092/

* You can browse recordings of past webinars at https://www.syslog-ng.com/events/ 

Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/


Peter Czanik (CzP) <peter.czanik at oneidentity.com>
Balabit (a OneIdentity company) / syslog-ng upstream
https://syslog-ng.com/community/
https://twitter.com/PCzanik