April 2023 - syslog-ng - lists.balabit.hu

using -dv on startup vs syslog-ng-ctl debug set=off
by Jerry Scharf 28 Apr '23

28 Apr '23

Hi, We are chasing an intermittent bug with slow reloads. We are following the debug suggestions and adding -dv to the execstart in the service file. Because of the load of messages that we push, I want to turn off the debugging after we are past the window when the slow reload happens. I naively tried doing "syslog-ng-ctl debug set=off" but it responded that the flag didn't change. Is there some way to tell syslog-ng to allow the -ctl command to make the change. I can imagine why this would be desired sometimes, but at 10k lines per minute it is going to fill up /tmp. right now I am doing ugly things to solve the problem, but really want to find a better way. thanks, jerry -- Jerry Scharf Pure Storage

2 1

What is the status of syslog-ng 4 in your environment?
by Peter Czanik (pczanik) 27 Apr '23

27 Apr '23

Hi, Version 4 of syslog-ng was released almost half a year ago. It adds type support to name-value pairs, and lots of improvements to Python and Kubernetes support. I just started a Twitter / Mastodon / LinkedIn poll to learn, what is the status of syslog-ng 4 in your environment? * https://fosstodon.org/@PCzanik/110269557855094446 * https://twitter.com/sngose/status/1651486235469201408 * https://www.linkedin.com/posts/peterczanik_poll-syslogabrng-python-activity… Of course, I'd also be happy to hear some feedback, not just a poll result. So, let me know why you are on syslog-ng 4 already, or why do you delay your upgrade to this new major version! If hearing about syslog-ng 4 completely surprised you, check https://www.syslog-ng.com/community/b/blog/posts/upgrade-problems-from-sysl… which describes some of the major changes, and what to be careful about when you upgrade! Peter Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Insider 2023-04: FreeBSD; Splunk; Deduplication;
by Peter Czanik (pczanik) 20 Apr '23

20 Apr '23

Dear syslog-ng users, This is the 109th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Installing a syslog-ng 4 development snapshot on FreeBSD -------------------------------------------------------- Unless there is a serious problem, FreeBSD ports usually contains the latest stable syslog-ng release. However, sometimes people want to compile a git snapshot to test a new feature or bugfix. To do that, one way is to generate a syslog-ng release tgz on FreeBSD and edit the syslog-ng port files yourself. However, this needs some practice. As such, an easier solution is to use my weekly development snapshots. https://www.syslog-ng.com/community/b/blog/posts/installing-a-syslog-ng-4-d… Getting data to Splunk ---------------------- Getting data to Splunk can be challenging. Syslog is still the most important data source, and it can provide you with hard-to-solve problems (for example, like high volume, non-compliant messages, unreliable network protocol (UDP), and more). The syslog-ng Premium Edition (PE) and syslog-ng Store Box (SSB) by One Identity can make these challenges manageable. https://www.syslog-ng.com/community/b/blog/posts/getting-data-to-splunk Streaming deduplication in syslog-ng ------------------------------------ Log volumes are growing 25% year over year, which means they are doubling every three years. Considering that SIEMs and other log processing tools are licensed based on volume, tools and mechanisms to make log storage and processing more efficient are very much sought for. Lean, how syslog-ng can help you. https://syslog-ng-future.blog/streaming-deduplication-in-syslog-ng/ WEBINARS * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Forwarding locally stored RAW messages
by Dragan Zecevic 13 Apr '23

13 Apr '23

Hi, we have syslog-ng Open Source Edition 3.33 and we are storing syslog messages from some systems into log files locally on a partition on syslog-ng server. Each day those log files are compressed. In order to better analyze some logs if needed we would like to extract some log files and ingest them to SIEM. Is it possible to make some forwarder that will read these RAW syslog messages from a log file and send them via syslog to SIEM? Thank you. Br, Dragan

2 1

TLS "trusted-dn" Question
by David Hauck 05 Apr '23

05 Apr '23

Hi, I'm currently using syslog-ng OSE v3.31.2 and trying to get a TLS configured endpoint to use the 'trusted-dn()' TLS option. I'm having trouble getting syslog-ng to return these DN specifiers in the Certificate Request option during the TLS negotiation so that clients can properly condition their supplied client certificates. I invariably see the following TLS negotiations (empty DNs list) in my Wireshark captures (as returned by the syslog-ng server): TLSv1.2 Record Layer: Handshake Protocol: Certificate Request Content Type: Handshake (22) Version: TLS 1.2 (0x0303) Length: 58 Handshake Protocol: Certificate Request Handshake Type: Certificate Request (13) Length: 54 Certificate types count: 3 Certificate types (3 types) Signature Hash Algorithms Length: 46 Signature Hash Algorithms (23 algorithms) Distinguished Names Length: 0 <----- always '0' In these cases my clients choose random client certificates that can't be refined to certificates signed by those expected (via the 'trusted-dn()' values) by the server and the connection is immediately closed. Here's the syslog-ng.conf entry for these sources: source s_515_tls { network( transport(tls) port(515) ip-protocol(6) tls(ca-dir("/etc/ssl/certs") key-file("/root/naservers.key") cert-file("/root/naservers.cer") peer-verify(required-trusted) trusted-dn("CN=*.netacquire.com")) ); }; I've tried several variants of the 'trusted-dn()' values, including other wildcards for country, state, etc. I always see a DNs list of zero size in the TLS Certificate Request option returned by the server. As expected switching to 'peer-verify(required-untrusted)' results in successful negotiation (with expected server-side errors indicating problems associated with the client certificates) and subsequent successful client/server logging. I figure I must be missing something obvious ;). Any ideas? Here's my syslog-ng version info: [logdest:~]# syslog-ng --version syslog-ng 3 (3.31.2) Config version: 3.29 Installer-Version: 3.31.2 Revision: Compile-Date: Nov 9 2021 12:52:59 Module-Directory: /usr/lib/syslog-ng Module-Path: /usr/lib/syslog-ng Include-Path: /usr/share/syslog-ng/include Available-Modules: tfgetent,mod-python,cryptofuncs,hook-commands,kvformat,cef,afuser,linux-kmsg-format,map-value-pairs,system-source,azure-auth-header,csvparser,affile,afprog,secure-logging,http,examples,timestamp,afsocket,confgen,stardate,dbparser,xml,disk-buffer,appmodel,afsnmp,add-contextual-data,afstomp,graphite,json-plugin,basicfuncs,syslogformat,pseudofile,tags-parser Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: on Enable-Linux-Caps: on Enable-Systemd: off Thanks, -David

3 10

Learning syslog-ng
by Peter Czanik (pczanik) 05 Apr '23

05 Apr '23

Hi, Last year, one of the returning questions I received was how to learn syslog-ng. My answer was that read the first few chapters of the documentation, read my blogs related to your use case, and then read a few relevant parts from the rest of the documentation. Our documentation is praised by users, but it is still a reference documentation. I was asked if a less detailed, more to the point, preferably video tutorial is available. Your request was heard. In the past couple of months, I published a tutorial series in blog and video format, which brings you from basic logging concepts to using syslog-ng to collect, parse, enrich log messages and store them to Elasticsearch. Of course, these 5-10 minute videos are not enough to learn anything in depth, but they introduce you to all major syslog-ng functionalities. Even if you are a seasoned syslog-ng user, there is a good chance that you will learn something new from this introductory tutorial series: the “if” statement, in-line configuration elements, the inlist() filter or the JSON template function, just to name a few. If you have roughly 1.5 hours, then I recommend going through all the videos from the beginning to the end. You can reach the play list on YouTube at: https://www.youtube.com/playlist?list=PLoBNbOHNb0i5Pags2JY6-6wH2noLaSiTb If you would rather pick only a few topics from the tutorial series, here is a table of contents, with short summaries, pointers to the blog and video versions and the related parts from the documentation. Unfortunately, the documentation for the latest version is not available yet, pointers are included to the web version of the syslog-ng version 3.37 documentation: https://peter.czanik.hu/posts/syslog-ng-tutorial-toc/ Of course, once you read/watched my syslog-ng tutorials, reading the blogs and relevant parts of the documentation is still highly recommended. Please share this news if you know anyone who wants to learn syslog-ng! Bye, CzP Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0