[syslog-ng] Insider 2021-10: OpenSearch; udp-balancer(); mqtt() destination; process accounting;

[Peter Czanik (pczanik)]

Dear syslog-ng users,

This is the 95th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news.

NEWS

Elasticsearch 7.14 and OpenSearch 1.0 are available – and work fine with syslog-ng
------------------------
One of the most popular destinations in syslog-ng is Elasticsearch. Due to the license change of the Elastic stack, some people changed quickly to Grafana/Loki and other technologies. However, most syslog-ng users decided to wait and see. Version 1.0.0 of OpenSearch, a fork of the Elastic code base from before the license change is now available. Elastic also published a new release last week. For this blog, I tested the latest and greatest from both product lines and I’m sharing my experiences. For the impatient: both work perfectly well.
https://www.syslog-ng.com/community/b/blog/posts/elasticsearch-7-14-and-opensearch-1-0-are-available-and-work-fine-with-syslog-ng

Using the udp-balancer() source of syslog-ng PE
-----------------------------------------------
UDP-based log collection is so last century. We had TCP-based log collection for decades and TLS encryption to secure connections. Still, UDP is in wide use, especially at large companies and industrial automation, where every change is slow. In most cases, UDP logging is used by networking devices, but sometimes it is just left there from ancient times and people are reluctant to change it. In either case, at higher message rates it can lead to performance problems and thus to message loss. Originally, the udp() source of syslog-ng was single-threaded. That does not scale well with typical multi-core CPUs with slower cores. There are many tricks to enhance UDP performance in syslog-ng. Combining those with the udp-balancer() source of syslog-ng PE gives the most reliable solution.
https://www.syslog-ng.com/community/b/blog/posts/using-the-udp-balancer-source-of-syslog-ng-pe

Syslog-ng 3.33: the MQTT destination
------------------------------------
Version 3.33 of syslog-ng introduced an MQTT destination. It uses the paho-c client library to send log messages to an MQTT broker. The current implementation supports version 3.1 and 3.1.1 of the protocol over non-encrypted connections, but this is only a first step. From this blog, you can learn how to configure and test the mqtt() destination in syslog-ng. Note: syslog-ng 3.34 enhanced the mqtt() destination in many ways.
https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-3-33-the-mqtt-destination

Collecting process accounting logs on Linux with syslog-ng
----------------------------------------------------------
Process accounting logs are collected into binary log files on Linux. You can turn them into human readable format locally, using various tools. You can also use syslog-ng to read those files. Syslog-ng can parse those binary logs, create name-value pairs from them and store the results.
https://www.syslog-ng.com/community/b/blog/posts/collecting-process-accounting-logs-on-linux-with-syslog-ng

WEBINARS

* You can browse recordings of past webinars at https://www.syslog-ng.com/events/

Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/


Peter Czanik (CzP) <peter.czanik at oneidentity.com>
Balabit (a OneIdentity company) / syslog-ng upstream
https://syslog-ng.com/community/
https://twitter.com/PCzanik