- syslog-ng - lists.balabit.hu

Missing Depends libjson-c4
by Joe Craig 06 Jul '22

06 Jul '22

The following packages have unmet dependencies: syslog-ng-core : Depends: libjson-c4 (>= 0.13.1) but it is not installable Depends: libssl1.1 (>= 1.1.1) but it is not installable >From research it looks like libjson-c4 has been replaced in the Debian repositories by libjson-c5 is there a way to utilize the libjson-c5 version or how should I resolve this issue? FYI this was working on Ubuntu 22.04 Jammy up until the past week. I ran updates on the system to resolve another issue and after that I'm getting an error starting the daemon × syslog-ng.service - System Logger Daemon Loaded: loaded (/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Tue 2022-07-05 15:39:48 UTC; 4s ago Docs: man:syslog-ng(8) Process: 1342 ExecStart=/usr/sbin/syslog-ng -F $SYSLOGNG_OPTS (code=exited, status=2) Main PID: 1342 (code=exited, status=2) Status: "Starting up... (Tue Jul 5 15:39:48 2022" CPU: 29ms Jul 05 15:39:48 <hostname> systemd[1]: syslog-ng.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Jul 05 15:39:48 <hostname> systemd[1]: syslog-ng.service: Failed with result 'exit-code'. Jul 05 15:39:48 <hostname> systemd[1]: Failed to start System Logger Daemon. Jul 05 15:39:48 <hostname> systemd[1]: syslog-ng.service: Scheduled restart job, restart counter is at 5. Jul 05 15:39:48 <hostname> systemd[1]: Stopped System Logger Daemon. Jul 05 15:39:48 <hostname> systemd[1]: syslog-ng.service: Start request repeated too quickly. Jul 05 15:39:48 <hostname> systemd[1]: syslog-ng.service: Failed with result 'exit-code'. Jul 05 15:39:48 <hostname> systemd[1]: Failed to start System Logger Daemon. The rabbit hole lead me to this current state. Any assistance would be appreciated. Joseph Craig

2 2

Environment variables in syslog
by Faisal Chishti 28 Jun '22

28 Jun '22

Hellow, Please see below the snippet from my syslog-ng.conf file. ABC_LOG_PATH is the environment variable on the host that contains the path. I am trying to access the env variable within the config. It does not throw any error and the configuration is valid but it does not work properly. I don't see the file agent.log being used at all. source s_agent { file("`ABC_LOG_PATH`/agent.log" follow-freq(1) flags(no-parse)); }; root@syslog-ng-784fdc497-pgdzr:/# syslog-ng --version syslog-ng 3 (3.30.1) Config version: 3.29 Installer-Version: 3.30.1 Revision: Compile-Date: Apr 12 2022 20:15:37 Module-Directory: /usr/lib/syslog-ng Module-Path: /usr/lib/syslog-ng Include-Path: /usr/share/syslog-ng/include Available-Modules: add-contextual-data,afamqp,affile,afprog,afsocket,afsql,afstomp,afuser,appmodel,azure-auth-header,basicfuncs,cef,confgen,cryptofuncs,csvparser,dbparser,disk-buffer,graphite,hook-commands,http,json-plugin,kvformat,linux-kmsg-format,map-value-pairs,mod-python,pseudofile,redis,secure-logging,stardate,syslogformat,system-source,tags-parser,timestamp,xml Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off Enable-Systemd: off root@syslog-ng-784fdc497-pgdzr:/# uname -a Linux syslog-ng-784fdc497-pgdzr 4.18.0-147.3.1.el8_1.x86_64 #1 SMP Fri Jan 3 23:55:26 UTC 2020 x86_64 GNU/Linux root@syslog-ng-784fdc497-pgdzr:/# -- Regards, Faisal

2 1

I am a novice on syslog-ng needs help.
by Alexandre Adao 20 Jun '22

20 Jun '22

Hello, I am a novice on syslog-ng. I need some assistance on installing syslog-ng and looking for a good recommendation on step-by-step instructions for Ubuntu (prefered OS) or Windows environment. I googled it but I My goal is to send the logs to splunk. Any help would be appreciated. Thanks, --Alex

3 3

Insider 2022-06: 3.37; hardware; Apache; disk-buffer;
by Peter Czanik (pczanik) 16 Jun '22

16 Jun '22

Dear syslog-ng users, This is the 102nd issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS syslog-ng 3.37 released ----------------------- Balázs Scheidler, founder of the syslog-ng project, describes the latest syslog-ng release: “syslog-ng 3.37 has just been released, packages available in various platforms this week. You can get the detailed release notes on the github releases page, however I felt this would be a good opportunity to revisit my draft on the syslog-ng long term objectives and how this release builds in that direction.” Read the rest of the blog at https://syslog-ng-future.blog/syslog-ng-3-37-released/ When (not) to use the syslog-ng disk-buffer ------------------------------------------- The syslog-ng disk buffer is one of the most often used syslog-ng options to ensure message delivery. However, it is not always necessary and using the safest variant has serious performance impacts. If you utilize disk-buffer in your syslog-ng configuration, it is worth to make sure that you use a recent syslog-ng version. >From this blog, you can learn when to use the disk-buffer option, the main differences between reliable and non-reliable disk-buffer, and why is it worth to use the latest syslog-ng version. https://www.syslog-ng.com/community/b/blog/posts/when-not-to-use-the-syslog… Hardware for a syslog-ng server ------------------------------- What hardware to use for a syslog-ng server? It is a frequent question with no definite answer. It depends on many factors: the number and type of sources, the number of logs, the way logs are processed, and so on. My experience is that for the majority users even a Raspberry Pi would be enough. But of course, not for everyone. https://www.syslog-ng.com/community/b/blog/posts/hardware-for-a-syslog-ng-s… Analyzing Apache HTTPD logs in syslog-ng ---------------------------------------- Recently, I started my own blog, and as Google Analytics seems to miss a good part of visitors, I wanted to analyze my web server logs myself. I use syslog-ng to read Apache logs, process them, and store them to Elasticsearch. Along the way, I resolve the IP address using a Python parser, analyze the Agent field of the logs, and also use GeoIP to locate the user on the map. >From this blog, you can learn how I built my configuration. Note that once I was ready, I realized that my configuration is not GDPR compliant, so I also show you which parts to remove from the final configuration :-). https://www.syslog-ng.com/community/b/blog/posts/analyzing-apache-httpd-log… WEBINARS * Register for our latest webinars at https://www.syslog-ng.com/event/syslogng-webcast-series-2022/ * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Test email.
by Tamás Pál 12 Jun '22

12 Jun '22

Please ignore this email.

1 0

Re: [syslog-ng] syslog-ng Digest, Vol 206, Issue 2
by Александр Масленников 10 Jun '22

10 Jun '22

i'm not sure that *format-flat-json* works with lists same as with nested dicts. My example uses format-flat-json, but without an additional filter on pothon, I was unable to flatten it There is original message { "operation": "CLIENT_RESPONSE", "identity": "ns-server.example.com", "family": "INET", "protocol": "TCP", "query-ip": "192.168.xx.zz", "query-port": "51000", "response-ip": "192.168.yy.zz", "response-port": "53", "length": 691, "rcode": "NOERROR", "qname": "_dnsaddr.bootstrap.libp2p.io", "qtype": "TXT", "latency": "0.000000", "timestamp-rfc3339": "2022-06-06T08:47:58.797332215Z", "answers": [{ "name": "_dnsaddr.bootstrap.libp2p.io", "rdatatype": "TXT", "ttl": 600, "rdata": "dnsaddr=/dnsaddr/ams-2.bootstrap.libp2p.io/p2p/QmbLHAnMoJPWSCR5Zhtx6BHJX9KiKNN6tpvbUcqanj75Nb" }, { "name": "_dnsaddr.bootstrap.libp2p.io", "rdatatype": "TXT", "ttl": 600, "rdata": "dnsaddr=/dnsaddr/sjc-1.bootstrap.libp2p.io/p2p/QmNnooDu7bfjPFoTZYxMNLWUQJyrVwtbZg5gBMjTezGAJN" }, { "name": "_dnsaddr.bootstrap.libp2p.io", "rdatatype": "TXT", "ttl": 600, "rdata": "dnsaddr=/dnsaddr/ewr-1.bootstrap.libp2p.io/p2p/QmQCU2EcMqAqQPR2i9bChDtGNJchTbq5TbXJJ16u19uLTa" }, { "name": "_dnsaddr.bootstrap.libp2p.io", "rdatatype": "TXT", "ttl": 600, "rdata": "dnsaddr=/dnsaddr/ams-rust.bootstrap.libp2p.io/p2p/12D3KooWEZXjE41uU4EL2gpkAQeDXYok6wghN7wwNVPF5bwkaNfS" }, { "name": "_dnsaddr.bootstrap.libp2p.io", "rdatatype": "TXT", "ttl": 600, "rdata": "dnsaddr=/dnsaddr/sjc-2.bootstrap.libp2p.io/p2p/QmZa1sAxajnQjVM8WjWXoMbmPd7NsWhfKsPkErzpm9wGkp" }, { "name": "_dnsaddr.bootstrap.libp2p.io", "rdatatype": "TXT", "ttl": 600, "rdata": "dnsaddr=/dnsaddr/nrt-1.bootstrap.libp2p.io/p2p/QmcZf59bWwK5XFi76CZX8cbJ4BhTzzA3gU1ZjYZcYW3dwt" }], "country-isocode": "-" } I would be very grateful to you if you have a solution using the built-in functions of syslog-ng. пт, 10 июн. 2022 г. в 15:00, <syslog-ng-request(a)lists.balabit.hu>: > Send syslog-ng mailing list submissions to > syslog-ng(a)lists.balabit.hu > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.balabit.hu/mailman/listinfo/syslog-ng > or, via email, send a message with subject or body 'help' to > syslog-ng-request(a)lists.balabit.hu > > You can reach the person managing the list at > syslog-ng-owner(a)lists.balabit.hu > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of syslog-ng digest..." > > > Today's Topics: > > 1. need help with parser to make flat nested json list of > dictionaries (????????? ???????????) > 2. Re: need help with parser to make flat nested json list of > dictionaries (Peter Kokai (pkokai)) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 10 Jun 2022 11:02:55 +0300 > From: ????????? ??????????? <alexander.a.maslennikov(a)gmail.com> > To: syslog-ng(a)lists.balabit.hu > Subject: [syslog-ng] need help with parser to make flat nested json > list of dictionaries > Message-ID: > <CA+G0nAjp1b6_50LbCROVPje1_B4R_AzNYiZ-_dT0m= > fXcqwmHA(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > hi all > i have a json message that contains a nested json list of dicts > > {"a":1,"b":[{"c":1},{"c":2},{"c":3}]} > > i want to flat that message, so expected result looks like { > "a": 1, > "b_0_c": 1, > "b_1_c": 2, > "b_2_c": 3 > } > > My approach is a python implemented parser. > Is it possible to achieve the same result using the built-in syslog-ng > tools? > My solution below > > @define kafka-implementation kafka-c > > python { > > import collections > import json > > class FlattenedJson(object): > > def parse(self, log_message, flat_message=None): > def flatten(d, parent_key='', sep='_'): > items = [] > for k, v in d.items(): > new_key = parent_key + sep + k if parent_key else k > if isinstance(v, collections.MutableMapping): > items.extend(flatten(v, new_key, sep=sep).items()) > elif isinstance(v, list): > for idx, value in enumerate(v): > items.extend(flatten(value, new_key + sep + > str(idx), sep).items()) > else: > items.append((new_key, v)) > return dict(items) > try: > decoded_msg = > json.loads(log_message['MESSAGE'].decode('utf-8')) > flat_message = flatten(decoded_msg) > final_message = > str(json.dumps(flat_message)).encode(encoding='utf-8') > log_message['MESSAGE'] = final_message > except Exception as error: > log_message['python_error'] = 'An exception occurred: > {}'.format(error) > return True > }; > > destination d_kafka_dnstap { > kafka( > topic("mytopic") > bootstrap-servers("localhost:9092") > message("$(format-flat-json --scope all-nv-pairs > application_name=myapp @timestamp=${ISODATE} )") > ); > }; > > source s_net_dnstap { network( transport(udp) port(514) flags(no-parse) ); > }; > > parser p_dnstap { channel { > parser { python(class("FlattenedJson")); }; > parser { json-parser(prefix("dnstap.")); }; > }; > }; > > log { source(s_net_dnstap); parser(p_dnstap); destination(d_kafka_dnstap); > }; >

2 1

need help with parser to make flat nested json list of dictionaries
by Александр Масленников 10 Jun '22

10 Jun '22

hi all i have a json message that contains a nested json list of dicts {"a":1,"b":[{"c":1},{"c":2},{"c":3}]} i want to flat that message, so expected result looks like { "a": 1, "b_0_c": 1, "b_1_c": 2, "b_2_c": 3 } My approach is a python implemented parser. Is it possible to achieve the same result using the built-in syslog-ng tools? My solution below @define kafka-implementation kafka-c python { import collections import json class FlattenedJson(object): def parse(self, log_message, flat_message=None): def flatten(d, parent_key='', sep='_'): items = [] for k, v in d.items(): new_key = parent_key + sep + k if parent_key else k if isinstance(v, collections.MutableMapping): items.extend(flatten(v, new_key, sep=sep).items()) elif isinstance(v, list): for idx, value in enumerate(v): items.extend(flatten(value, new_key + sep + str(idx), sep).items()) else: items.append((new_key, v)) return dict(items) try: decoded_msg = json.loads(log_message['MESSAGE'].decode('utf-8')) flat_message = flatten(decoded_msg) final_message = str(json.dumps(flat_message)).encode(encoding='utf-8') log_message['MESSAGE'] = final_message except Exception as error: log_message['python_error'] = 'An exception occurred: {}'.format(error) return True }; destination d_kafka_dnstap { kafka( topic("mytopic") bootstrap-servers("localhost:9092") message("$(format-flat-json --scope all-nv-pairs application_name=myapp @timestamp=${ISODATE} )") ); }; source s_net_dnstap { network( transport(udp) port(514) flags(no-parse) ); }; parser p_dnstap { channel { parser { python(class("FlattenedJson")); }; parser { json-parser(prefix("dnstap.")); }; }; }; log { source(s_net_dnstap); parser(p_dnstap); destination(d_kafka_dnstap); };

2 1

syslog-ng-3.37.1
by Laszlo Varady (lvarady) 08 Jun '22

08 Jun '22

3.37.1 Highlights * kubernetes source: A new source for Kubernetes CRI (Container Runtime Interface) format. By default it tails the /var/log/containers folder which can be overriden with the base-dir() parameter. Example configuration: source { kubernetes(); # or specifying the directory: # kubernetes(base-dir("/dir/to/tail")); }; (#4015<https://github.com/syslog-ng/syslog-ng/pull/4015>) * mariadb-audit-parser: A new parser for mariadb/mysql audit plugin logs have been added. The parser supports the syslog output type's format, see mariadb page<https://mariadb.com/kb/en/mariadb-audit-plugin> for details. (#3947<https://github.com/syslog-ng/syslog-ng/pull/3947>) Features * internal(): add rcptid tag to all trace messages that relate to incoming log messages. This makes it easier to correlate parsing, rewriting and routing actions with incoming log messages. (#3972<https://github.com/syslog-ng/syslog-ng/pull/3972>) * syslog-parser(): allow comma (e.g. ',') to separate the seconds and the fraction of a second part as some devices use that character. This change applies to both to syslog-parser() and the builtin syslog parsing functionality of network source drivers (e.g. udp(), tcp(), network() and syslog()). (#3949<https://github.com/syslog-ng/syslog-ng/pull/3949>) * cisco-parser: add ISO 8601 timestamp support (#3934<https://github.com/syslog-ng/pull/3934>) * network(), syslog() sources and destinations: added new TLS options sigalgs() and client-sigalgs() They can be used to restrict which signature/hash pairs can be used in digital signatures. It sets the "signature_algorithms" extension specified in RFC5246 and RFC8446. Example configuration: destination { network("test.host" port(4444) transport(tls) tls( pkcs12-file("/path/to/tls/test.p12") peer-verify(yes) sigalgs("RSA-PSS+SHA256:ed25519") ) ); }; (#4000<https://github.com/syslog-ng/syslog-ng/pull/4000>) * set-matches() and unset-matches(): these new rewrite operations allow the setting of match variables ($1, $2, ...) in a single operation, based on a syslog-ng list expression. Example: # set $1, $2 and $3 respectively set-matches("foo,bar,baz"); # likewise, but using a list function set-matches("$(explode ':' 'foo:bar:baz')"); (#3948<https://github.com/syslog-ng/syslog-ng/pull/3948>) * $* macro: the $* macro in template expressions convert the match variables (e.g. $1, $2, ...) into a syslog-ng list that can be further manipulated using the list template functions, or turned into a list in type-aware destinations. (#3948<https://github.com/syslog-ng/syslog-ng/pull/3948>) * set-tag(): add support for using template expressions in set-tag() rewrite operations, which makes it possible to use tag names that include macro references. (#3962<https://github.com/syslog-ng/syslog-ng/pull/3962>) Bugfixes * http() and other threaded destinations: fix $SEQNUM processing so that only local messages get an associated $SEQNUM, just like normal syslog()-like destinations. This avoids a [meta sequenceId="XXX"] SD-PARAM being added to $SDATA for non-local messages. (#3928<https://github.com/syslog-ng/syslog-ng/pull/3928>) * grouping-by(): fix grouping-by() use through parser references. Originally if a grouping-by() was part of a named parser statement and was referenced from multiple log statements, only the first grouping-by() instance behaved properly, 2nd and subsequent references were ignoring all configuration options and have reverted using defaults instead. (#3957<https://github.com/syslog-ng/syslog-ng/pull/3957>) * db-parser(): similarly to grouping-by(), db-parser() also had issues propagating some of its options to 2nd and subsequent references of a parser statement. This includes drop-unmatched(), program-template() and template() options. (#3957<https://github.com/syslog-ng/syslog-ng/pull/3957>) * match(), subst() and regexp-parser(): fixed storing of numbered (e.g. $1,$2, $3 and so on) and named capture groups in regular expressions in case the input of the regexp is the same as one of the match variables being stored. In some cases the output of the regexp was clobbered and an invalid value stored. (#3948<https://github.com/syslog-ng/syslog-ng/pull/3948>) * fix threaded(no) related crash: if threaded mode is disabled for asynchronous sources and destinations (all syslog-like drivers such as tcp/udp/syslog/network qualify), a use-after-free condition can happen due to a reference counting bug in the non-threaded code path. The threaded(yes) setting has been the default since 3.6.1 so if you are using a more recent version, you are most probably unaffected. If you are using threaded(no) a use-after-free condition happens as the connection closes. The problem is more likely to surface on 32 bit platforms due to pointer sizes and struct layouts where this causes a NULL pointer dereference. (#3997<https://github.com/syslog-ng/syslog-ng/pull/3997>) * set(): make sure that template formatting options (such as time-zone() or frac-digits()) are propagated to all references of the rewrite rule containing a set(). Previously the clone() operation used to implement multiple references missed the template related options while cloning set(), causing template formatting options to be set differently, depending on where the set() was referenced from. (#3962<https://github.com/syslog-ng/syslog-ng/pull/3962>) * csv-parser(): fix flags(strip-whitespace) and null-value handling for greedy column (#4028<https://github.com/syslog-ng/syslog-ng/pull/4028>) Other changes * java()/python() destinations: the $SEQNUM macro (and "seqnum" attribute in Python) was erroneously for both local and non-local logs, while it should have had a value only in case of local logs to match RFC5424 behavior (section 7.3.1). This bug is now fixed, but that means that all non-local logs will have $SEQNUM set to zero from this version on, e.g. the $SEQNUM macro would expand to an string, to match the syslog() driver behaviour. (#3928<https://github.com/syslog-ng/syslog-ng/pull/3928>) * dbld: add support for Fedora 35 in favour of Fedora 33 (#3933<https://github.com/syslog-ng/syslog-ng/pull/3933>) * debian: fix logrotate file not doing the file rotation. (The path and command was invalid.) (#4031<https://github.com/syslog-ng/syslog-ng/pull/4031>) * OpenSSL: add support for OpenSSL 3.0 (#4012<https://github.com/syslog-ng/syslog-ng/pull/4012>) * The MD4 hash function ($(md4)) is no longer available when compiling syslog-ng with OpenSSL v3.0. MD4 is now deprecated, it will be removed completely in future versions. (#4012<https://github.com/syslog-ng/syslog-ng/pull/4012>) Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Andras Mitzki, Attila Szakacs, Balazs Scheidler, Ben Burrows, Fᴀʙɪᴇɴ Wᴇʀɴʟɪ, Gabor Nagy, László Várady, mohitvaid, Parrag Szilárd, Peter Kokai, Peter Viskup, Roffild, Ryan Faircloth, Scott Parlane, Zoltan Pallagi Installation from binaries: https://github.com/syslog-ng/syslog-ng#installation-from-binaries<https://github.com/syslog-ng/syslog-ng#installation-from-binaries>

1 0

rhel 9 / centos 9
by Peter Czanik (pczanik) 19 May '22

19 May '22

Hi, Testing and feedback needed! RHEL9 was released recently. My syslog-ng git snapshot repo now also supports RHEL 9. I tested my syslog-ng packages on CentOS Stream 9. Let me know, how it works for you! It is available at: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng-githead/ Note, that even if SQL support (libdbi) is enabled, the drivers are not there, I could not get them compiled. I plan to remove SQL support next week, as without drivers it's not much use. Peter Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

HUP or reload
by Faine, Mark R. (MSFC-IS40)[AEGIS] 19 May '22

19 May '22

Are there certain circumstances where I should prefer doing a HUP on the syslog-ng process instead of syslog-ng-ctl reload? My use case specifically is for postrotate but I'm also asking generally. Thanks, Mark

2 1

Insider 2022-05: directions; EPEL 9; throttle; regexp-parser;
by Peter Czanik (pczanik) 18 May '22

18 May '22

Dear syslog-ng users, This is the 101st issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS syslog-ng on the long term: a draft on strategic directions ----------------------------------------------------------- Balázs Scheidler, founder of the syslog-ng project, describes five possible strategic directions: “In the past few weeks I performed a round of discussions/interviews with syslog-ng users. I also spent time looking at other products and analyst reports on the market. Based on all this information I’ve come up with a list of potential strategic directions for syslog-ng to tackle. Focusing on these and prioritizing features that fall into one of these directions ensures that syslog-ng indeed moves ahead.” Read the rest of the blog at https://syslog-ng-future.blog/syslog-ng-on-the-long-term-a-draft-on-strateg… A minimalist syslog-ng package is heading to EPEL 9 --------------------------------------------------- Last week, the ivykis library, the most important core dependency of syslog-ng landed in EPEL 9 successfully. There are still plenty of dependencies missing, but this way, I could submit a slightly cut down version of syslog-ng to EPEL 9. Hopefully the rest of the dependencies will arrive in EPEL 9 as well. I plan to update the syslog-ng package as soon as the dependencies arrive. Luckily, these are only needed to enable some less frequently used syslog-ng destination drivers, no core functionality is affected. https://www.syslog-ng.com/community/b/blog/posts/a-minimalist-syslog-ng-pac… The difference between throttle() and rate-limit() in syslog-ng --------------------------------------------------------------- There are multiple ways in syslog-ng to limit message rate. The throttle() option of syslog-ng destinations tries to make sure that all messages are delivered without exceeding a specified message rate. The rate-limit() filter introduced in syslog-ng 3.36 drops surplus log messages, making sure that a processing pipeline or destination is not overloaded with log messages. https://www.syslog-ng.com/community/b/blog/posts/the-difference-between-thr… Using the regexp-parser of syslog-ng ------------------------------------ For many years, you could use the match() filter of syslog-ng to parse log messages with regular expressions. However, the primary function of match() is filtering. Recent syslog-ng versions now have a dedicated regular expression parser, the regexp-parser(). So, you should use match() only if your primary use case is filtering. Otherwise, use the regexp-parser for parsing, as it is a lot more flexible. https://www.syslog-ng.com/community/b/blog/posts/using-the-regexp-parser-of… WEBINARS * Register for our latest webinars at https://www.syslog-ng.com/event/syslogng-webcast-series-2022/ * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Re: [syslog-ng] Not logging postfix logs in mail.info mail.warn (SysAdmin EM)
by Szilard Parrag (sparrag) 17 May '22

17 May '22

Hi emawata, Checking your configuration snippet it seems that `d_mailwarn` and `d_mailinfo` are not connected to a logpath, which is why syslog-ng doesn't write into them. If you wish to use them create a `log{};` block, which fetches data from source and writes it into `destination(d_mailwarn);` or `destination(d_mailinfo);` Hope it helped, Szilard

1 0

Re: [syslog-ng] Local sources seem not to be working
by Szilard Parrag (sparrag) 16 May '22

16 May '22

Hi Alex, After checking the stats, you have sent we can see that there had been some writes: from dst.program;d_localfile_linecard#0;/opt/machine/local/bin/write_with_rotation.sh /var/log/linecard.log 10 10;a;written;4518 to dst.program;d_localfile_linecard#0;/opt/machine/local/bin/write_with_rotation.sh /var/log/linecard.log 10 10;a;written;4549 * we do not see increase in the counters of /var/log destinatons, but only on one destination * we could only see an increase in syslog-udp processed counters * there are no dropped/queued counters We would guess this could be due to flow-control, but for that we would need to see non-zero queued counter values, which is not the case. It could happen that one destination hangs/can't send messages out, which leads to suspended sources due to flow-control, but the syslog() source is not affected since it doesn't send messages to the hanged destination(s). Based on the stats, only "d_localfile_linecard" is active (~30 messages in 15 minutes), maybe the syslog() source would be affected too without the filtering. We should see more internal logs, which is problematic, since internal() source seems to be stopped too. For that I would recommend extracting internal() source from the s_src statement and putting it in a separate log path with a simple file destination. Also, if possible, could you please share your `write_with_rotation.sh` script? It is unlikely that it interferes with syslog-ng, but a double check would be nice. 🙂

2 2

Not logging postfix logs in mail.info mail.warn
by SysAdmin EM 16 May '22

16 May '22

I cannot configure Postfix logs correctly, the mail.warn and mail.info logs are empty. Any helps?? Here is my configuration: options { chain_hostnames(off); flush_lines(0); use_dns(no); use_fqdn(no); owner("root"); group("adm"); perm(0640); stats_freq(0); bad_hostname("^gconfd$"); }; source s_src { system(); internal(); }; destination d_auth { file("/var/log/auth.log"); }; destination d_cron { file("/var/log/cron.log"); }; destination d_daemon { file("/var/log/daemon.log"); }; destination d_kern { file("/var/log/kern.log"); }; destination d_lpr { file("/var/log/lpr.log"); }; destination d_mail { file("/var/log/mail.log"); }; destination d_syslog { file("/var/log/syslog"); }; destination d_user { file("/var/log/user.log"); }; destination d_uucp { file("/var/log/uucp.log"); }; destination d_mailinfo { file("/var/log/mail.info"); }; destination d_mailwarn { file("/var/log/mail.warn"); }; destination d_mailerr { file("/var/log/mail.err"); }; destination d_newscrit { file("/var/log/news/news.crit"); }; destination d_newserr { file("/var/log/news/news.err"); }; destination d_newsnotice { file("/var/log/news/news.notice"); }; destination d_debug { file("/var/log/debug"); }; destination d_error { file("/var/log/error"); }; destination d_messages { file("/var/log/messages"); }; destination d_console { usertty("root"); }; destination d_console_all { file(`tty10`); }; destination d_xconsole { pipe("/dev/xconsole"); }; log_fifo_size(1000)); }; destination d_ppp { file("/var/log/ppp.log"); }; filter f_dbg { level(debug); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_err { level(err); }; filter f_crit { level(crit .. emerg); }; filter f_debug { level(debug) and not facility(auth, authpriv, news, mail); }; filter f_error { level(err .. emerg) ; }; filter f_messages { level(info,notice,warn) and not facility(auth,authpriv,cron,daemon,mail,news); }; filter f_auth { facility(auth, authpriv) and not filter(f_debug); }; filter f_cron { facility(cron) and not filter(f_debug); }; filter f_daemon { facility(daemon) and not filter(f_debug); }; filter f_kern { facility(kern) and not filter(f_debug); }; filter f_lpr { facility(lpr) and not filter(f_debug); }; filter f_local { facility(local0, local1, local3, local4, local5, local6, local7) and not filter(f_debug); }; filter f_mail { facility(mail) and not filter(f_debug); }; filter f_news { facility(news) and not filter(f_debug); }; filter f_syslog3 { not facility(auth, authpriv, mail) and not filter(f_debug); }; filter f_user { facility(user) and not filter(f_debug); }; filter f_uucp { facility(uucp) and not filter(f_debug); }; filter f_cnews { level(notice, err, crit) and facility(news); }; filter f_cother { level(debug, info, notice, warn) or facility(daemon, mail); }; filter f_ppp { facility(local2) and not filter(f_debug); }; filter f_console { level(warn .. emerg); }; ######################## # Log paths ######################## log { source(s_src); filter(f_auth); destination(d_auth); }; log { source(s_src); filter(f_cron); destination(d_cron); }; log { source(s_src); filter(f_daemon); destination(d_daemon); }; log { source(s_src); filter(f_kern); destination(d_kern); }; log { source(s_src); filter(f_lpr); destination(d_lpr); }; log { source(s_src); filter(f_syslog3); destination(d_syslog); }; log { source(s_src); filter(f_user); destination(d_user); }; log { source(s_src); filter(f_uucp); destination(d_uucp); }; log { source(s_src); filter(f_mail); destination(d_mail); }; log { source(s_src); filter(f_news); filter(f_crit); destination(d_newscrit); }; log { source(s_src); filter(f_news); filter(f_err); destination(d_newserr); }; log { source(s_src); filter(f_news); filter(f_notice); destination(d_newsnotice); }; log { source(s_src); filter(f_debug); destination(d_debug); }; log { source(s_src); filter(f_error); destination(d_error); }; log { source(s_src); filter(f_messages); destination(d_messages); }; log { source(s_src); filter(f_console); destination(d_console_all); destination(d_xconsole); }; log { source(s_src); filter(f_crit); destination(d_console); };

1 0

Filtering all systemd user service logs
by Thomas Preisner 02 May '22

02 May '22

Hello all, I've been using syslog-ng for quite a while now to centrally collect logs of a few computers running debian and it worked great. However, a few weeks ago I enabled systemd user services on those machines and since then, my 'daemon' logs have been basically completely flooded with (in my opinion) useless log messages created by the aforementioned user services. So far, I've been using the following source configuration: source src { internal(); system(); udp(); udp6(); }; Until now, I've been filtering out the user service logs with my log-analysis tool using some regexes but due to the logs being 'user-controlled' (users can create arbitrary logs by simply creating and starting custom systemd user services) this is obviously no permanent solution. Even worse, the user-created logs end up in the 'daemon'-facility and not in the 'user'-facility as mentioned before. Therefore, here my question: Is there any clean way to detect/filter out these user service logs? I couldn't find anything relevant in the documentation so far. After all, these logs are collected separately by systemd-journald and then merged with the remaining system logs by syslog-ng (at least as far as I see it). Thank you for your response in advance! Best regards, Thomas Preisner

2 1

Local sources seem not to be working
by Alexandre Santos 28 Apr '22

28 Apr '22

Hi, I have syslog-ng 3.32.1 running in a Debian GNU/Linux 10 (buster) with the configuration in the attachement. After sometime running, syslog-ng seems be unable to read from system() and internal() sources. Log messages from syslog(ip(10.20.30.40) transport("udp") port(514) keep-alive(no)); are seen in the output folders. Also journald logs are working fine. After a reload of configuration in which what changes is this line: rewrite r_host { set("MACHINE-${HOST}", value("HOST")); }; logging is resumed. Here is the time gap for logs: <43>1 2022-03-11T11:55:23.802+00:00 xmm4-1-1 syslog-ng 8283 - [meta sequenceId="767"] Last message 'Destination reliable' repeated 8933 times, suppressed by syslog-ng on xmm4-1-1 <46>1 2022-03-14T07:19:01.817+00:00 xmm4-1-1 syslog-ng 8283 - [meta sequenceId="1"] Module loaded and initialized successfully; module='syslogformat' Do you know why this is happening? Thanks & Regards, Alex

2 13

Insider 2022-04: typing; sudo; Zinc; Elastic Cloud; 3.36;
by Peter Czanik (pczanik) 21 Apr '22

21 Apr '22

Dear syslog-ng users, This is the 100th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS syslog-ng 4 theme: typing ------------------------- Balázs Scheidler, founder of the syslog-ng project, describes a major new syslog-ng version 4.0 feature: typing. “syslog-ng traditionally assumes that log data, even if it comes in a structured form (like RFC5424 structured data or JSON) is primarily textual in nature. For this reason, name-value pairs in syslog-ng are text values just as the log message as a whole.” Read the rest of the blog to learn how typing helps not just when forwarding logs to Elasticsearch and others, but also when doing routing decisions: https://syslog-ng-future.blog/syslog-ng-4-theme-typing/ Working with JSON logs from sudo in syslog-ng --------------------------------------------- >From this blog you will learn how to work with JSON formatted logs in syslog-ng and also about new sudo features along the way: JSON logging in sudo, chroot support, logging sub-commands, and how to work with these logs in syslog-ng. https://www.syslog-ng.com/community/b/blog/posts/working-with-json-logs-fro… Elasticsearch 8 and syslog-ng ----------------------------- General availability of Elasticsearch 8 was announced recently. There were quite a few rumors that it will break compatibility with third party tools. I tested it as soon as I had a little time: I am happy to share that anything I tested with the elasticsearch-http() destination of syslog-ng still seems to work perfectly well with the latest version of Elasticsearch. https://www.syslog-ng.com/community/b/blog/posts/elasticsearch-8-and-syslog… Contacting the syslog-ng team: reporting problems, asking questions --------------------------------------------- Recently I got some complaints that it is difficult to figure out how to contact the syslog-ng team to get help or report problems. Most of this information is available both on the syslog-ng website and at the syslog-ng repository on GitHub, but collecting here all information might be still useful for some people. https://www.syslog-ng.com/community/b/blog/posts/contacting-the-syslog-ng-t… WEBINARS * Register for our latest webinars at https://www.syslog-ng.com/event/syslogng-webcast-series-2022/ * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

[GSoC] Rotate log files based on size
by Devharsh Trivedi 13 Apr '22

13 Apr '22

Hi @all, I am a PhD student and would like to apply to the Google Summer of Code. The proposal "Rotate log files based on size” seems interesting to me. Please advise. Regards, Devharsh.

3 2

3.36.1 new log format of some internal messages
by Evan Rempel 06 Apr '22

06 Apr '22

syslog-ng version 3.36.1 has introduced an additional time stamp in some of its internally generated log messages. [2022-04-05T07:10:27.574110] WARNING: Configuration file format is too old, syslog-ng is running in compatibility mode. Please update it to use the syslog-ng 3.36 format at your time of convenience. To upgrade the configuration, please review the warnings about incompatible changes printed by syslog-ng, and once completed change the @version header at the top of the configuration file; config-version='3.34' In the past this log message would have been formatted as WARNING: Configuration file format is too old, syslog-ng is running in compatibility mode. Please update it to use the syslog-ng 3.36 format at your time of convenience. To upgrade the configuration, please review the warnings about incompatible changes printed by syslog-ng, and once completed change the @version header at the top of the configuration file; config-version='3.34' Is this intentional? Why is it not consistent with other internal messages such as syslog-ng starting up; version='3.36.1' Another thing of note is that the configuration warning is actually logged via journald while all other internal messages are only available in the syslog-ng log streams sourcing "internal()" -- Evan

2 2

syslog-ng and ELK integration
by Park, Ethan 04 Apr '22

04 Apr '22

Greetings, I don't know this is the right place to ask about this question but I will give it a shot. We are running a syslog-ng server and we want to use Kibana as a visualization tool. I got stuck in at this point when configuring conf.d file. source s_tcp { tcp(ip("0.0.0.0") port("514")); udp(ip("0.0.0.0") port("514")); }; destination d_el { elasticsearch2( client-mode("http") index("syslog-ng") type("test") cluster-url(http://10.1.3.222:9200) ); #file("/var/log/networks/$HOST/$YEAR/$MONTH/$HOST-$YEAR-$MONTH-$DAY.log"); }; log { source(s_tcp); destination(d_el); flags(flow-control); }; If I start syslog-ng daemon, I got the below error. Apr 04 11:54:42 syslog-ng syslog-ng[22294]: [2022-04-04T11:54:42.443764] Plugin module not found in 'module-path'; module-path='/usr/lib/syslog-ng/3.13', module='mod-java' Apr 04 11:54:42 syslog-ng syslog-ng[22294]: Error parsing destination, destination plugin elasticsearch2 not found in /etc/syslog-ng/conf.d/firewals.conf at line 10, column Apr 04 11:54:42 syslog-ng syslog-ng[22294]: included from /etc/syslog-ng/syslog-ng.conf line 163, column 1 Apr 04 11:54:42 syslog-ng syslog-ng[22294]: elasticsearch2( Apr 04 11:54:42 syslog-ng syslog-ng[22294]: ^^^^^^^^^^^^^^ Apr 04 11:54:42 syslog-ng syslog-ng[22294]: syslog-ng documentation: https://www.balabit.com/support/documentation?product=syslog-ng-ose Apr 04 11:54:42 syslog-ng syslog-ng[22294]: contact: https://lists.balabit.hu/mailman/listinfo/syslog-ng Apr 04 11:54:42 syslog-ng systemd[1]: syslog-ng.service: Main process exited, code=exited, status=1/FAILURE Apr 04 11:54:42 syslog-ng systemd[1]: syslog-ng.service: Failed with result 'exit-code'. Apr 04 11:54:42 syslog-ng systemd[1]: Failed to start System Logger Daemon. -- Subject: Unit syslog-ng.service has failed I am running 3.13 (syslog-ng) and what did I miss? PRIVILEGED AND CONFIDENTIAL COMMUNICATION This message is privileged and confidential under state and federal law. It is intended for the exclusive use of the named recipient. If the reader of this message is not the named or intended recipient, or the employee or agent responsible for delivering the message to its intended recipient, you are hereby notified that any use, copying, disclosure or dissemination of this message is strictly prohibited. If you are not the named or intended recipient and have received this communication in error, please notify us immediately by replying to the sender of this e-mail or at the address and phone number listed below, and take all steps necessary to delete this communication. Exactech, Inc. 2320 NW 66th Ct. Gainesville, FL. 32653 352-377-1140 An Equal Opportunity Employer

2 1

Multiple options blocks
by Faine, Mark R. (MSFC-IS40)[NICS] 02 Apr '22

02 Apr '22

Can I have an options block in syslog-ng.conf that is for default configuration such as for /var/log and another options block in an include file that is specific to those destinations?

2 1

Permissions are not set correctly on created directories
by Faine, Mark R. (MSFC-IS40)[NICS] 28 Mar '22

28 Mar '22

I have the following in my syslog-ng.conf for options: options { stats_freq(60); flush_lines(500); #flush_timeout(10000); time_reopen(10); time_reap(5); log_fifo_size(536870912); chain_hostnames(off); log-msg-size(65536); use_dns(no); keep_hostname(yes); use_fqdn(yes); dns_cache(off); dns-cache-size(2000); dns-cache-expire(87600); create_dirs(yes); perm(0640); owner(root); group(splunk); dir-perm(2750); dir-owner(root); dir-group(splunk); threaded(yes); }; However, instead of 2750 for the directory permissions, I'm seeing d-wSrwxrwT. 10 root splunk 4096 Mar 28 14:07 ise [root@molecule8-mfaine data]# stat -c '%a' ise 5276 I would expect: drwxr-s---. 2 root splunk 4096 Mar 28 14:11 ise Where are these permissions coming from? Thanks, Mark

2 2

Value is dropped or unset in resolved destination template
by Faine, Mark R. (MSFC-IS40)[NICS] 25 Mar '22

25 Mar '22

I have a destination template in my destination (d_default) that uses $location destination d_default { file("`BASEPATH`/$location/$(lowercase ${HOST})/$app/$(lowercase ${HOST})_$app.log" create_dirs(yes) flags("threaded", "no-multi-line")); }; That is set by a series of filters that look like this log { if { filter... rewrite { set("foo" value("location")); }; }; if { filter... rewrite { set("bar" value("location")); }; }; ... }; This appears to be working, I can see it in the logs: Setting value; name='location', value='unknown', msg='0x7f190c001e00' # Unknown is a default value that would be appropriate on this testing server Later additional log paths are applied like so: log { source(default); filter(f_asa); rewrite(r_asa); destination(d_default); flags(final ); }; ... log { source(default); filter(f_fw); rewrite(r_fw); destination(d_default); flags(final ); }; ... However, when I go to the destination on the filesystem the location is empty so the files are being written as though location was this: ("`BASEPATH`//$(lowercase ${HOST})/$app/$(lowercase ${HOST})_$app.log" Any ideas what could be causing this? Thanks, Mark

4 5

Help with embedded log paths
by Faine, Mark R. (MSFC-IS40)[NICS] 25 Mar '22

25 Mar '22

I need this configuration to do as the comments say but I can't figure out how to make it happen. Any pointers would be appreciated. Thanks, Mark log { source(default); # One of these first four should always match, if not $location=unknown log { filter(f_arc); rewrite(r_arc); }; log { filter(f_gsfc); rewrite(r_gsfc); }; log { filter(f_jsc); rewrite(r_jsc); }; log { filter(f_msfc); rewrite(r_msfc); }; ## The first one of these to match writes to disk and stops processing further log paths ## I don't know how this can be done since I can't add flags(final) inside of an embedded log path # Log Path for asa log { filter(f_asa); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase asa/${HOST}/${HOST}-asa.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for fmg log { filter(f_fmg); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase fmg/${HOST}/${HOST}-fmg.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for faz log { filter(f_faz); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase faz/${HOST}/${HOST}-faz.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for fw log { filter(f_fw); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase fw/${HOST}/${HOST}-fw.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for fw_block log { filter(f_fw_block); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase fw_block/${HOST}/${HOST}-fw_block.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for fortigate log { filter(f_fortigate); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase fortigate/${HOST}/${HOST}-fortigate.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for ips log { filter(f_ips); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase ips/${HOST}/${HOST}-ips.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for psa log { filter(f_psa); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase psa/${HOST}/${HOST}-psa.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for pubfw log { filter(f_pubfw); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase pubfw/${HOST}/${HOST}-pubfw.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for utm_other log { filter(f_utm_other); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase utm_other/${HOST}/${HOST}-utm_other.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for waf log { filter(f_waf); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase waf/${HOST}/${HOST}-waf.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; # Log Path for webfilter log { filter(f_webfilter); destination { file("/var/log/remote/data/${location:-unknown}/$(lowercase webfilter/${HOST}/${HOST}-webfilter.${UNIXTIME}.log)" create_dirs(yes) flags("threaded", "no-multi-line")); }; }; flags(final); };

3 4

Customizing syslog-ng snmp() destination option
by Maurya, Shivani 23 Mar '22

23 Mar '22

Hi All, I am using syslog-ng v3.30 and forwarding snmp traps from this server using snmp() destination option - [cid:image001.png@01D839F6.A0D8F220] I wanted to add a field "agent-addr" as part of the trap & not in the variable-binding. This field "agent-addr" will contain the IP address of the device which is sending the syslog message to the syslog server. How can I add a customized filed as part of the trap? Regards, Shivani Maurya

4 9

Gsoc preparation
by Kavinda Shehan Rajapakshe 22 Mar '22

22 Mar '22

Hello, I’m Kavinda Rajapakse, an undergraduate at the University of Moratuwa, Sri Lanka. Currently, I'm studying Computer Science and Engineering. This is my first time contributing to open source. I have some experience working in Rsyslog and Elastic stack. I would like to contribute to the "Support Elastic's beats protocol” project in GSOC 2022. I would be highly obliged if someone could provide me with an introduction as to how to get started. Thank you.

2 1

Insider 2022-03: syslog-ng 4; MQTT source; Zinc; Elastic Cloud; 3.36;
by Peter Czanik (pczanik) 17 Mar '22

17 Mar '22

Dear syslog-ng users, This is the 99th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS syslog-ng future: the path to syslog-ng 4 ----------------------------------------- Balázs Scheidler, founder of the syslog-ng project, describes the path to syslog-ng version 4.0: “It is very important to state that syslog-ng 4 is not the revolutionary, break-everything kind of release that we see too often in the software world. Rather it is an evolutionary change that will be produced similarly to previous releases” Read the rest of the blog at https://syslog-ng-future.blog/syslog-ng-future-the-path-to-syslog-ng-4/ MQTT source ----------- Version 3.35.1 of syslog-ng introduced an MQTT source. Just for some fun in the last syslog-ng blog post of the year, I created an endless loop using syslog-ng and the Mosquitto MQTT broker. Of course, it does not have much practical value other than possibly a bit of stress testing, but hopefully provides a fun introduction to MQTT-related technologies in syslog-ng. https://www.syslog-ng.com/community/b/blog/posts/creating-an-endless-loop-u… Another use for the syslog-ng elasticsearch-http destination: Zinc --------------------------- There is a new drop-in replacement for Elasticsearch, at least if you don’t mind the limitations and the alpha status. However, it definitely lives up to the promise that it provides an Elasticsearch-compatible API for data ingestion. I tested it with the elasticsearch-http() destination of syslog-ng, and it worked perfectly after I modified the URL in the configuration example I found. https://www.syslog-ng.com/community/b/blog/posts/another-use-for-the-syslog… Sending logs to Elastic Cloud using syslog-ng --------------------------------------------- The Elastic Cloud is a service by Elastic providing Elasticsearch and related services in an easy-to-use package. Last year someone reported an issue that it does not work properly with syslog-ng. I did not have time to investigate at that time. Now I started a free trial and soon my log messages from syslog-ng started to appear in Kibana in Elastic Cloud. >From this blog you can learn how to configure syslog-ng for the Elastic Cloud. I go with the most basic settings: exploring Elastic Cloud and syslog-ng Elasticsearch features are both out of scope, as both are very well documented on their respective websites. NEW RELEASES * syslog-ng 3.36: https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-3-36-news-better… WEBINARS * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

my syslog-ng-stable repos are updated to 3.36.1
by Peter Czanik (pczanik) 16 Mar '22

16 Mar '22

Hi, Version 3.36.1 of syslog-ng was released almost two weeks ago. Hundreds of people already installed it, nobody reported any problems, so my syslog-ng-stable RPM repos for openSUSE / SLES and Fedora / RHEL and compatibles are now updated to syslog-ng 3.36.1 See https://www.syslog-ng.com/community/b/blog/posts/introducing-the-syslog-ng-… Peter Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

where did you switch from CentOS 8 Linux?
by Peter Czanik (pczanik) 16 Mar '22

16 Mar '22

Hi, By now I hope anyone who used CentOS 8 Linux proper previously switched to something else. If the question in the subject of my e-mail surprises you, please read the announcement about making CentOS 8 Linux End of Life by the end of 2021: https://blog.centos.org/2020/12/future-is-centos-stream/ By now all repositories are removed as well. Question: Where did you switch? RHEL or a compatible release by Rocky Linux, AlmaLinux? CentOS 8 Stream, the rolling pre-release of RHEL? Or something completely different, like openSUSE or FreeBSD? You can answer here or on my Twitter poll: https://twitter.com/PCzanik/status/1504034771944493056 Background: For years roughly 80% of syslog-ng users ran RHEL / CentOS & compatibles. This is expected to change now. Some stay on traditional RHEL (& compatible) releases, some move to Stream, some to something completely different. Your answer helps me to decide where to focus my effort as a syslog-ng packager. Peter Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

a question about the elasticsearch-http driver
by Risto Vaarandi 12 Mar '22

12 Mar '22

hi all, I have a question about the elasticsearch-http driver. Is it possible to send events to a specific Elasticsearch ingest pipeline with this driver? Also, if sending events to ingest pipeline is possible, can the ingest pipeline name contain macros, so that the pipeline name can be set based on the data extracted from incoming events? Such functionality is implemented in rsyslog, and I was wondering if the same thing can be accomplished with syslog-ng. kind regards, risto

2 3

parsing cisco firepower logs problem with 3.33
by Stoffel, John (TAI) 09 Mar '22

09 Mar '22

Hi, I'm trying to parse some cisco logs from a Cisco firepower firewall, using syslog-ng v3.33 on a CentOS 7 system. After pounding my head against the wall a few times to realize that you can't just re-start syslog-ng and have it re-read a source file from scratch... that instead I need to just push the data using netcat, it's now in a state where I think I can try to debug things. My logs look like this: <166>2022-02-16T15:31:53Z na-zy-int-fp1140-p02 : %FTD-6-305012: Teardown dynamic UDP translation fr om TAI-INSIDE:1.2.3.110/51288 to FOO-OUTSIDE:6.7.8.18/33333 duration 0:00:00 <166>2022-02-16T15:31:53Z na-zy-int-fp1140-p02 : %FTD-6-305012: Teardown dynamic TCP translation fr om FOO-WAN_IN:10.92.60.80/59877 to FOO-OUTSIDE:6.7.8.18/59877 duration 0:01:01 <166>2022-02-16T15:31:53Z na-zy-int-fp1140-p02 : %FTD-6-305011: Built dynamic UDP translation from FOO-INSIDE:1.2.3.110/51288 to FOO-OUTSIDE:6.7.8.18/5632 Looking at this log, vs the examples given in the /usr/share/syslog-ng/include/scl/cisco/plugin.conf file, I think the problem is that my logs shows the: sequence, date: origin, %MSG instead of sequence, origin, date: %MSG and it's not clear to me how I would hack the plugin.conf file to handle this issue. My end goal is to be able to parse the message enough by log level so I can forward only a subset of messages to another remote syslog system. Thanks, John Sr. Storage Architect TOSHIBA AMERICA, INC. 1251 6th, Ave 41st flr, New York, NY 10020 508-736-5499 (mobile) E-Mail: john.stoffel(a)toshiba.com<mailto:john.stoffel@toshiba.com> Website: Service Now Self Service Portal<https://nassc.service-now.com/ess/navpage.do>

2 7

syslog-ng-3.36.1
by Laszlo Varady (lvarady) 04 Mar '22

04 Mar '22

3.36.1 Highlights * system() source: added basic support for reading macOS system logs The current implementation processes the output of the original macOS syslogd: /var/log/system.log. (#3710<https://github.com/syslog-ng/syslog-ng/pull/3710>) * $(values) and $(names): these new template functions can be used to query a list of name-value pairs in the current message. The list of name value pairs queried are specified by a value-pairs expression, just like with $(format-json). Examples: This expression sets the JSON array values to contain the list of SDATA values, while the JSON array names would contain the associated names, in the same order. $(format-json values=list($(values .SDATA.*)) names=list($(names .SDATA.*))) The resulting name-value pairs are always sorted by their key, regardless of the argument order. (#3911<https://github.com/syslog-ng/syslog-ng/pull/3911>) * rename(): added a new rewrite rule, called rename() Example usage: rewrite { rename( "renamed-from" "renamed-to" ); }; (#3841<https://github.com/syslog-ng/syslog-ng/pull/3841>) Features * network() drivers: added TLS keylog support syslog-ng dumps TLS secrets for a given source/destination, which can be used for debugging purposes to decrypt data with, for example, Wireshark. This should be used for debugging purposes only! Example usage: source tls_source{ network( port(1234) transport("tls"), tls( key-file("/path/to/server_key.pem"), cert-file("/path/to/server_cert.pem"), ca-dir("/path/to/ca/") keylog-file("/path/to/keylog_file") ) ); }; (#3792<https://github.com/syslog-ng/syslog-ng/pull/3792>) * tls() block: added option for restricting TLS 1.3 ciphers The network(), syslog(), and the http() modules now support specifying TLS 1.3 cipher suites, Example usage: network( transport("tls") tls( pkcs12-file("test.p12") cipher-suite( tls12-and-older("ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256"), tls13("TLS_CHACHA20_POLY1305_SHA256:TLS_AES_256_GCM_SHA384") ) ) ); tls12-and-older() can be used to specify TLS v1.2-and-older ciphers, tls13() can be used for TLS v1.3 ciphers only. Note: The old cipher-suite("list:of:ciphers") option restricts only the TLS v1.2-and-older cipher suite for backward compatibility. (#3907<https://github.com/syslog-ng/syslog-ng/pull/3907>) * file() destination: added a new option: symlink-as() This feature allows one to maintain a persistent symlink to a log file when a template is used (for example: /var/log/cron -> /var/log/cron.${YEAR}${MONTH}). Example usage: destination d_file_cron { file("/var/log/cron.${YEAR}${MONTH}" symlink-as("/var/log/cron")); }; >From a functional perspective, the symlink-as file inherits both create-dirs and file ownership from its file destination (permissions are not applicable to symlinks, at least on linux). The symlink is adjusted at the time a new destination file is opened (in the example above, if ${YEAR} or ${MONTH} changes). Although not specific to time macros, that's where the usefulness is. If the template contains something like ${PROGRAM} or ${HOST}, the configuration wouldn't necessarily be invalid, but you'd get an ever-changing symlink of dubious usefulness. (#3855<https://github.com/syslog-ng/syslog-ng/pull/3855>) * flags(no-rfc3164-fallback): added a new flag to sources that parse incoming syslog data and operate in RFC5424 mode (e.g. syslog-protocol is also set). With the new flag the automatic fallback to RFC3164 format is disabled. In this case if the parsing in RFC5424 fails, the syslog parser would result in an error message. In the case of syslog-parser(drop-invalid(yes)), the message would be dropped. (#3891<https://github.com/syslog-ng/syslog-ng/pull/3891>) * syslog-format: accept ISO timestamps that incorrectly use a space instead of a 'T' to delimit the date from the time portion. For example, a "2021-01-01T12:12:12" timestamp is well formed according to RFC5424 (which uses a subset of ISO8601, see https://datatracker.ietf.org/doc/html/rfc5424#section-6.2.3) Some systems simply use a space instead of a 'T'. The same format is accepted for both RFC3164 (e.g. udp(), tcp() and network() sources) and RFC5424 (e.g. syslog() source). (#3893<https://github.com/syslog-ng/syslog-ng/pull/3893>) * transport(text-with-nuls): added a new transport mechanism for the network() driver that allows NUL characters within the message. Note: syslog-ng does not support embedded NUL characters everywhere, so it is recommended that you also use flags(no-multi-line) that causes NUL characters to be replaced by space. (#3913<https://github.com/syslog-ng/syslog-ng/pull/3913>) Bugfixes * filter: fixed the not operator in filter expressions (regression in v3.35.1) Reusing a filter that contains the not operator more than once, or referencing a complex expression containing not might have caused invalid results in the previous syslog-ng version (v3.35.1). This has been fixed. (#3863<https://github.com/syslog-ng/syslog-ng/pull/3863>) * throttle() filter: support negation (#3863<https://github.com/syslog-ng/syslog-ng/pull/3863>) * disk-buffer(): fixed a crash which could happen in very rare cases, while a corrupted disk-buffer was getting replaced (#3845<https://github.com/syslog-ng/syslog-ng/pull/3845>) * disk-buffer(): fixed a memory leak issue and inconsistent buffer handling in rare cases (#3887<https://github.com/syslog-ng/syslog-ng/pull/3887>) * disk-buffer(): fixed underflowing queued stats counter (#3887<https://github.com/syslog-ng/syslog-ng/pull/3887>) * disk-buffer(): fixed queued stats were not adjusted when a disk-buffer became corrupt (#3851<https://github.com/syslog-ng/syslog-ng/pull/3851>) * disk-buffer(): fixed a disk-buffer corruption issue A completely filled and then emptied disk-buffer may have been recognised as corrupt. (#3874<https://github.com/syslog-ng/syslog-ng/pull/3874>) * amqp(): fixed a minor error reporting problem. (#3869<https://github.com/syslog-ng/syslog-ng/pull/3869>) * amqp(): syslog-ng now drops messages that are too large to send (#3869<https://github.com/syslog-ng/syslog-ng/pull/3869>) * amqp(): fixed a crash, which happened with librabbitmq v0.9.0 or v0.10.0, while using the tls() block. (#3929<https://github.com/syslog-ng/syslog-ng/pull/3929>) * file() source: fixed invalid buffer handling when encoding() is used A bug has been fixed that - under rare circumstances - could cause message duplication or partial message loss when non-fixed length or less known fixed-length encodings are used. (#3892<https://github.com/syslog-ng/syslog-ng/pull/3892>) * syslog-ng: fixed a SIGSEGV triggered by an incorrectly formatted "CONFIG" command, received on the syslog-ng control socket. The only known implementation of the control protocol is syslog-ng-ctl itself, which always sends a correct command, but anyone with access to the UNIX domain socket syslog-ng.ctl (root only by default) can trigger a crash. (#3900<https://github.com/syslog-ng/syslog-ng/pull/3900>) * credit-card-mask(): fixed visa, mastercard and jcb card regex pattern (#3853<https://github.com/syslog-ng/syslog-ng/pull/3853>) * cisco-parser(): allow a leading dot in the timestamp (not synced clocks) (#3843<https://github.com/syslog-ng/syslog-ng/pull/3843>) Notes to developers * plugins: we have made it easier to implement filter plugins An example can be found under modules/rate-limit-filter. (#3866<https://github.com/syslog-ng/syslog-ng/pull/3866>) * dev-utils: various fixes for the plugin skeleton generator script (#3866<https://github.com/syslog-ng/syslog-ng/pull/3866>) Other changes * The syslog-ng Docker image<https://hub.docker.com/r/balabit/syslog-ng/> is now automatically tagged and pushed to Docker Hub after each release (#3870<https://github.com/syslog-ng/syslog-ng/pull/3870>) * throttle() filter: renamed to rate-limit() (#3866<https://github.com/syslog-ng/syslog-ng/pull/3866>) * python: support Python 3.10 (#3865<https://github.com/syslog-ng/syslog-ng/pull/3865>) * java: upgraded from old log4j v1.x line to log4j v2.17.2 (#3861<https://github.com/syslog-ng/syslog-ng/pull/3861>) (#3927<https://github.com/syslog-ng/syslog-ng/pull/3927>) Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Andras Mitzki, Andrea Biardi, Attila Szakacs, Balazs Scheidler, Balázs Barkó, Benedek Cserhati, Gabor Nagy, Janos SZIGETVARI, Laszlo Budai, Laszlo Szemere, László Várady, Mikel Olasagasti Uranga, Norbert Takacs, Parrag Szilárd, Peter Kokai, Szilárd Parrag, Zoltan Pallagi, Stanislav Osipov, Yash Mathne <https://github.com/syslog-ng/syslog-ng#installation-from-binaries>Installation from binaries https://github.com/syslog-ng/syslog-ng#installation-from-binaries

1 0

graylog2 plugin not found
by Daniel Estermann 03 Mar '22

03 Mar '22

Hi Fabien, that's it, thank you very much for the quick response! Kinda clairvoyant btw ;-) Best regards Daniel Am 03.03.22, 13:24 schrieb "syslog-ng im Auftrag von Fabien Wernli" <syslog-ng-bounces(a)lists.balabit.hu im Auftrag von wernli(a)in2p3.fr>: Hi Daniel, Could you show your full config ? I think you might be missing the following directive: @include "scl.conf" [http://www.ti8m.com/dam/jcr:59611d49-12ae-460f-b117-37384c0a05c7] <https://www.ti8m.com/insights/downloads/special-security>

1 0

graylog2 plugin not found
by Daniel Estermann 03 Mar '22

03 Mar '22

Hi everybody, I am running RHEL 8.5 ➜ ~ head -2 /etc/os-release NAME="Red Hat Enterprise Linux" VERSION="8.5 (Ootpa)" ➜ ~ uname -a Linux zhm-lm00 4.18.0-348.12.2.el8_5.x86_64 #1 SMP Mon Jan 17 07:06:06 EST 2022 x86_64 x86_64 x86_64 GNU/Linux The installation of syslog-ng was performed via DNF using EPEL repository: ➜ ~ dnf install -y syslog-ng >8.......8< Installing: syslog-ng x86_64 3.23.1-2.el8 epel 842 k Installing dependencies: ivykis x86_64 0.42.4-2.el8 epel 47 k libnet x86_64 1.1.6-15.el8 rhel-8-for-x86_64-appstream-rpms 67 k >8.......8< Now, I was trying to configure transport of syslog to graylog using the following documentation: https://www.syslog-ng.com/community/b/blog/posts/graylog-destination-syslog… However, I've received the following error message, when i tried to start syslog-ng with the new configuration: Error parsing destination statement, destination plugin graylog2 not found in /etc/syslog-ng/syslog-ng.conf:9:5-9:13: 4 }; 5 6 filter f_gitlab_runner {"${.SDATA.journald._SYSTEMD_UNIT}" eq "gitlab-runner.service"}; 7 8 destination d_graylog { 9-----> graylog2( 9-----> ^^^^^^^^ 10 host("[MASKED]") 11 port(12201) 12 transport(tcp) 13 ); 14 }; syslog-ng documentation: https://www.balabit.com/support/documentation?product=syslog-ng-ose contact: https://lists.balabit.hu/mailman/listinfo/syslog-ng Do I have to install that plugin separately? How to do that? Kind Regards Daniel Estermann [http://www.ti8m.com/dam/jcr:59611d49-12ae-460f-b117-37384c0a05c7] <https://www.ti8m.com/insights/downloads/special-security>

2 1

Syslog-ng v3.31 destination option not working
by Maurya, Shivani 25 Feb '22

25 Feb '22

Hi, I am referring the syslog-ng admin guide - https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edi… and using the snmp() destination option. The following is included in my syslog-ng.conf file as part of destination - destination d_snmpv2c{ snmp( version('v2c') host('192.168.1.1') trap-obj('.1.3.6.1.6.3.1.1.4.1.0', 'Objectid', '.1.3.6.1.4.1.18372.3.1.1.1.2.1') snmp-obj('.1.3.6.1.4.1.18372.3.1.1.1.1.1.0', 'Octetstring', 'Test SNMP trap') snmp-obj('.1.3.6.1.4.1.18372.3.1.1.1.1.2.0', 'Octetstring', 'admin') snmp-obj('.1.3.6.1.4.1.18372.3.1.1.1.1.3.0', 'Ipaddress', '192.168.1.1') ); }; But syslog-ng service/daemon is not starting & pointing error at snmp(). Can you please help on this? FYI - I am using RHEL7. Regards, Shivani Maurya

3 5

allowed concurrent connections - bug?
by Evan Rempel 18 Feb '22

18 Feb '22

I am having an issue that is a little difficult to reproduce so I wanted some input from others. I have a syslog-ng 3.35.1 that has a TLS source defined with max-connections(10000) After some time the server starts logging a lot of messages syslog-ng[12802]: Number of allowed concurrent connections reached, rejecting connection; client='AF_INET(XXXX:61062)', local='AF_INET(YYYY:6514)', group_name='client_network_tcp', location='/etc/syslog-ng/syslog-ng.server.conf:61:9', max='10000' To the best of my ability I can only find about 2500 actual connections. Both lsof and netstat report around the 2500 connections. I had to restart syslog-ng to stop this situation. Has anyone seen this behavior before? I get a lot of TLS connections without a certificate. Error reading RFC6587 style framed data Pperhaps the counters are not decremented for those timed out connections? -- Evan Rempel

2 5

Yocto: build syslog-ng with Python support
by Christian Hollermann 16 Feb '22

16 Feb '22

Hi, in our project we use a Yocto based build system (branch dunfell, Linux Kernel 5.10) for a while. At the next step, we plan to capture log messages from different sources with the help of syslog-ng (3.24.1). For some sources we will use a Python script that implements an interface to another system. Our problem is now, that we cannot build syslog-ng with a working Python support. Our build completes successfully, but the newly generated library 'libmod-python.so' seems to be broken as we get an error message when we use the following command to verify the installation. syslog-ng --module-registry | grep python Error opening plugin module; module='mod-python', error='/usr/lib/syslog-ng/libmod-python.so: undefined symbol: PyExc_ValueError' Module: mod-python Status: Unable to dlopen shared object, probably not a syslog-ng module Below you can see our current syslog recipe. syslog-ng_%.bbappend: FILESEXTRAPATHS_append := ":${THISDIR}/${PN}" DEPENDS += " python3 " # Override of default config file SRC_URI += "file://syslog-ng.conf.systemd" # Just to be sure EXTRA_OECONF_remove = "--disable-python" # Enable Python Support EXTRA_OECONF += "--enable-python --with-python=3" Can you give us hint, how to to fix this behavior? Thanks and Regards, Christian

1 0

syslog-ng OSE v3.30.1 - slowdown when using env variable in destination clause
by Wilson, Jonathan L 14 Feb '22

14 Feb '22

I have a pair of syslog-ng OSE servers on different LANs, which forward some of the log entries that they receive to separate collectors. In other words, Syslog-A -> collector-A Syslog-B -> collector-B I would like for the two syslog servers to use a common syslog-ng.conf for obvious reasons. I first built destination clauses with the separate IPs, like so: destination d_collector { udp( "192.168.1.23" port(514) template("${DATE} ${HOST} ${MSG}") ); }; This worked fine. But when I define an environment variable, COLLECTOR_IP, and then change my configuration to: destination d_collector { udp( "`COLLECTOR_IP`" port(514) template("${DATE} ${HOST} ${MSG}") ); }; Then syslog-ng's throughput slows down to a trickle (but does continue); CPU utilization (ordinarily substantial - this is a busy log handler) drops to near zero. Changing the configurations back to use the raw IP address restored previous throughput. There are no unusual messages in syslog-ng's own logfile. Anyone seen this puzzling behavior? Thanks, Jon Wilson jonathan.wilson(a)vumc.org<mailto:jonathan.wilson@vumc.org>

2 1

syslog-ng 3.35.1-2.el7
by Evan Rempel 10 Feb '22

10 Feb '22

The packaged syslog-ng RPM for RHEL7 from the stable copr repo https://copr-be.cloud.fedoraproject.org/results/czanik/syslog-ng-stable fails to have a dependency specified for glib library with a large enough version to satisfy the runtime requirements. The syslog-ng package has a requirement listed of "libglib-2.0.so.0()(64bit)" but if there is an older glib2 package installed that satisfies this requirement syslog-ng will fail to start with an error /usr/sbin/syslog-ng: symbol lookup error: /usr/lib64/libsyslog-ng-3.35.so.0: undefined symbol: g_ptr_array_find_with_equal_func Can the package maintainer add a >= version specification to the requirements so that the install of syslog-ng will automatically update to a version of glib2 that works with syslog-ng? -- Evan Rempel

4 5

Insider 2022-01: Reboot; Sequence; Monterey; CentOS 9;
by Peter Czanik (pczanik) 10 Feb '22

10 Feb '22

Dear syslog-ng users, This is the 98th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS syslog-ng relaunch ------------------ Balázs Scheidler, founder of the syslog-ng project, started a new blog where he details why and how he started to work on syslog-ng even more actively. He also asks for your feedback! “syslog-ng has been around for decades: I started coding the first version of syslog-ng in September 1998, circa 24 years ago. The adoption of syslog-ng skyrocketed soon after that: people installed it in place of the traditional syslogd across the globe. It was packaged for Debian, Gentoo, SUSE and even commercial UNIXes. It became a default logging daemon in some of these Linux distributions. Commercial products started embedding it as a system component. Over the years however I feel that syslog-ng has become a trusted piece of infrastructure, few people really care about. I set out to change that.” Read the rest of the blog at https://syslog-ng-future.blog/syslog-ng-relaunch/ Sequence – making PatternDB creation for syslog-ng easier --------------------------------------------------------- We are well into the 21st century, but most of the log messages still arrive in an unstructured format. For well over a decade, syslog-ng had a solution to turn unstructured messages into name-value pairs, called PatternDB. However, creating a pattern database for PatternDB from scratch is a source of major pain. Or rather, it was: sequence-rtg – a fork of the sequence log analyzer – provides a new hope! It can easily create ready-to-use patterns for your most frequent log messages. Sequence-rtg is still in beta phase, and therefore is a bit rough around the edges. However, once you deal with the initial struggles of creating the database, it works just fine. Especially if you have lots of log messages. My experience was that the more log messages and larger batch sizes I had, the better quality patterns were generated. https://www.syslog-ng.com/community/b/blog/posts/sequence-making-patterndb-… Syslog-ng on MacOS Monterey --------------------------- Each new MacOS release brings some surprises when it comes to compiling syslog-ng. Just a couple of months ago, I provided you with a couple of pointers on how to compile syslog-ng on MacOS. Since then, MacOS Monterey was released and Homebrew was updated. So, here are some updated instructions for MacOS Monterey. https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-on-macos-monterey Installing syslog-ng on CentOS Stream 9 --------------------------------------- CentOS Stream 9 has been around for a while, but it was officially announced just a few days ago. I already tested some earlier snapshots and they had some rough edges. The current version installed without random crashes, has networking and runs smoothly. EPEL – the semi-official repository by Fedora maintainers – is already there, but practically empty, syslog-ng or it’s dependencies are not yet there. As someone asked about syslog-ng support, I had a first try at building it. https://www.syslog-ng.com/community/b/blog/posts/installing-syslog-ng-on-ce… WEBINARS * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Re: [syslog-ng] destination plugin syslog-ng not found
by Clayton Dukes 13 Jan '22

13 Jan '22

Interesting. I didn't work until I added syslog-ng-mod-extra Clayton Dukes CEO, LogZilla Corporation <http://www.linkedin.com/in/lzcdukes/> <http://twitter.com/logzilla> (855) LogZilla <tel:(855)%20LogZilla> 9106.CDUKES <tel:(910)%20623-8537> cdukes(a)logzilla.net <mailto:cdukes@logzilla.net> logzilla.net <http://logzilla.net/> 4819 Emperor Blvd., Suite 400, Raleigh, NC, 27703 <https://calendly.com/teamneo/logzilla-neo-discovery> Schedule a 15 minute demo <https://calendly.com/teamneo/logzilla-neo-discovery> <https://youtu.be/yGQqkXegdoo> LZ in 4 Minutes <https://youtu.be/yGQqkXegdoo> LogZilla Corporation delivers a Centralized Log Management (CLM) platform, designed for IT Operations, Security, and Risk Management leaders who want to gain better incident investigation capabilities by capturing all network and security related logs into a single log collection platform. On 1/12/22, 2:24 AM, "syslog-ng on behalf of Peter Kokai (pkokai)" <syslog-ng-bounces(a)lists.balabit.hu on behalf of Peter.Kokai(a)oneidentity.com> wrote: Hello, The syslog-ng-core should contain both syslog and syslog-ng destination plugin. -- Kokan ________________________________________ From: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu> on behalf of Clayton Dukes <cdukes(a)logzilla.net> Sent: 12 January 2022 01:36 To: Syslog-ng users' and developers' mailing list Subject: [syslog-ng] destination plugin syslog-ng not found CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Hi All, Which Debian/apt module do I need for the `syslog-ng` destination plugin? # apt-cache search syslog-ng syslog-ng - Enhanced system logging daemon (metapackage) syslog-ng-core - Enhanced system logging daemon (core) syslog-ng-dbg - Enhanced system logging daemon (debug symbols) syslog-ng-dev - Enhanced system logging daemon (development files) syslog-ng-mod-add-contextual-data - Enhanced system logging daemon (add-contextual-data plugin) syslog-ng-mod-amqp - Enhanced system logging daemon (AMQP plugin) syslog-ng-mod-examples - Enhanced system logging daemon (example plugins) syslog-ng-mod-extra - Enhanced system logging daemon (extra plugins) syslog-ng-mod-geoip2 - Enhanced system logging daemon (GeoIP2 plugin) syslog-ng-mod-getent - Enhanced system logging daemon (getent plugin) syslog-ng-mod-graphite - Enhanced system logging daemon (graphite plugin) syslog-ng-mod-http - Enhanced system logging daemon (HTTP destination) syslog-ng-mod-map-value-pairs - Enhanced system logging daemon (map-value-pairs plugin) syslog-ng-mod-mongodb - Enhanced system logging daemon (MongoDB plugin) syslog-ng-mod-python - Enhanced system logging daemon (Python plugin) syslog-ng-mod-rdkafka - Enhanced system logging daemon (Kafka destination, based on librdkafka) syslog-ng-mod-redis - Enhanced system logging daemon (Redis plugin) syslog-ng-mod-riemann - Enhanced system logging daemon (Riemann destination) syslog-ng-mod-slog - Enhanced system logging daemon (secure logging plugin) syslog-ng-mod-smtp - Enhanced system logging daemon (SMTP plugin) syslog-ng-mod-snmp - Enhanced system logging daemon (SNMP plugin) syslog-ng-mod-sql - Enhanced system logging daemon (SQL plugin) syslog-ng-mod-stardate - Enhanced system logging daemon (stardate plugin) syslog-ng-mod-stomp - Enhanced system logging daemon (STOMP plugin) syslog-ng-mod-xml-parser - Enhanced system logging daemon (xml parser plugin) [photo] [signature_870575490] Clayton Dukes CEO, LogZilla Corporation [signature_1853374605]<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.linked…> [signature_1525617044]<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Ftwitter.co…> [signature_470467322] (855) LogZilla<tel:(855)%20LogZilla> [signature_1064870479] 9106.CDUKES<tel:(910)%20623-8537> [signature_288996986] cdukes(a)logzilla.net<mailto:cdukes@logzilla.net> [signature_2057347955] logzilla.net<https://nam12.safelinks.protection.outlook.com/?url=http%3A%2F%2Flogzilla.n…> [signature_518161642] 4819 Emperor Blvd., Suite 400, Raleigh, NC, 27703 [signature_1426204243]<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcalendly.…> Schedule a 15 minute demo<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fcalendly.…> [signature_1135219367]<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fyoutu.be%…> [signature_1977302009]LZ in 4 Minutes<https://nam12.safelinks.protection.outlook.com/?url=https%3A%2F%2Fyoutu.be%…> LogZilla Corporation delivers a Centralized Log Management (CLM) platform, designed for IT Operations, Security, and Risk Management leaders who want to gain better incident investigation capabilities by capturing all network and security related logs into a single log collection platform. [Logo Description automatically generated] IMPORTANT NOTICE: This e-mail message is intended to be received only by persons entitled to receive the confidential information it may contain. E-mail messages to clients of Logzilla Corporation may contain information that is confidential and legally privileged. Please do not read, copy, forward, or store this message unless you are an intended recipient of it. If you have received this message in error, please forward it to the sender and delete it completely from your computer system. ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq IMPORTANT NOTICE: This e-mail message is intended to be received only by persons entitled to receive the confidential information it may contain. E-mail messages to clients of Logzilla Corporation may contain information that is confidential and legally privileged. Please do not read, copy, forward, or store this message unless you are an intended recipient of it. If you have received this message in error, please forward it to the sender and delete it completely from your computer system.

3 6

destination plugin syslog-ng not found
by Clayton Dukes 12 Jan '22

12 Jan '22

Hi All, Which Debian/apt module do I need for the `syslog-ng` destination plugin? # apt-cache search syslog-ng syslog-ng - Enhanced system logging daemon (metapackage) syslog-ng-core - Enhanced system logging daemon (core) syslog-ng-dbg - Enhanced system logging daemon (debug symbols) syslog-ng-dev - Enhanced system logging daemon (development files) syslog-ng-mod-add-contextual-data - Enhanced system logging daemon (add-contextual-data plugin) syslog-ng-mod-amqp - Enhanced system logging daemon (AMQP plugin) syslog-ng-mod-examples - Enhanced system logging daemon (example plugins) syslog-ng-mod-extra - Enhanced system logging daemon (extra plugins) syslog-ng-mod-geoip2 - Enhanced system logging daemon (GeoIP2 plugin) syslog-ng-mod-getent - Enhanced system logging daemon (getent plugin) syslog-ng-mod-graphite - Enhanced system logging daemon (graphite plugin) syslog-ng-mod-http - Enhanced system logging daemon (HTTP destination) syslog-ng-mod-map-value-pairs - Enhanced system logging daemon (map-value-pairs plugin) syslog-ng-mod-mongodb - Enhanced system logging daemon (MongoDB plugin) syslog-ng-mod-python - Enhanced system logging daemon (Python plugin) syslog-ng-mod-rdkafka - Enhanced system logging daemon (Kafka destination, based on librdkafka) syslog-ng-mod-redis - Enhanced system logging daemon (Redis plugin) syslog-ng-mod-riemann - Enhanced system logging daemon (Riemann destination) syslog-ng-mod-slog - Enhanced system logging daemon (secure logging plugin) syslog-ng-mod-smtp - Enhanced system logging daemon (SMTP plugin) syslog-ng-mod-snmp - Enhanced system logging daemon (SNMP plugin) syslog-ng-mod-sql - Enhanced system logging daemon (SQL plugin) syslog-ng-mod-stardate - Enhanced system logging daemon (stardate plugin) syslog-ng-mod-stomp - Enhanced system logging daemon (STOMP plugin) syslog-ng-mod-xml-parser - Enhanced system logging daemon (xml parser plugin) [photo] [signature_870575490] Clayton Dukes CEO, LogZilla Corporation [signature_1853374605]<http://www.linkedin.com/in/lzcdukes/> [signature_1525617044]<http://twitter.com/logzilla> [signature_470467322] (855) LogZilla<tel:(855)%20LogZilla> [signature_1064870479] 9106.CDUKES<tel:(910)%20623-8537> [signature_288996986] cdukes(a)logzilla.net<mailto:cdukes@logzilla.net> [signature_2057347955] logzilla.net<http://logzilla.net/> [signature_518161642] 4819 Emperor Blvd., Suite 400, Raleigh, NC, 27703 [signature_1426204243]<https://calendly.com/teamneo/logzilla-neo-discovery> Schedule a 15 minute demo<https://calendly.com/teamneo/logzilla-neo-discovery> [signature_1135219367]<https://youtu.be/yGQqkXegdoo> [signature_1977302009]LZ in 4 Minutes<https://youtu.be/yGQqkXegdoo> LogZilla Corporation delivers a Centralized Log Management (CLM) platform, designed for IT Operations, Security, and Risk Management leaders who want to gain better incident investigation capabilities by capturing all network and security related logs into a single log collection platform. [Logo Description automatically generated] IMPORTANT NOTICE: This e-mail message is intended to be received only by persons entitled to receive the confidential information it may contain. E-mail messages to clients of Logzilla Corporation may contain information that is confidential and legally privileged. Please do not read, copy, forward, or store this message unless you are an intended recipient of it. If you have received this message in error, please forward it to the sender and delete it completely from your computer system.

2 1

laszlo_budai's GPG key expired
by Szmytka, Piotr 03 Jan '22

03 Jan '22

Hi, It seems the GPG key used to sign Debian 10 packages has expired. The deb repository is therefore unusable. Can this be fixed please? $ curl -s http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… | gpg - gpg: WARNING: no command supplied. Trying to guess what you mean ... pub rsa2048 2017-08-25 [SC] [expired: 2021-12-31] F20F51628D04901AD01175013B92A8D27CFDAEDD uid home:laszlo_budai:syslog-ng OBS Project <home:laszlo_budai:syslog-ng@build.opensuse.org> Thanks! --- PIOTR SZMYTKA Senior Systems Engineer, Cloud Engineering piotr.szmytka(a)vimn.com<mailto:piotr.szmytka@vimn.com> VIMN Poland sp. z o.o. ul. Mokotowska 19, 00-560 Warsaw, POLAND [signature_1140942668]

2 1

Insider 2021-12: Humio; Log Management; Panther;
by Peter Czanik (pczanik) 16 Dec '21

16 Dec '21

Dear syslog-ng users, This is the 97th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Sending logs to Panther using syslog-ng --------------------------------------- Panther is an open-source log management system, which is also available as a service for a time-limited trial. It is still in beta phase, but it looks promising. You can see the “beta” sign on its opening page: https://app.panther.support/ I tested the time-limited cloud service version, but you can also install it locally, either from Dockerhub, or you can build the containers locally from the source. Even if it is still in beta phase, Panther comes with detailed documentation. There is a notable exception: while syslog-ng is shown on some of the figures, documenting it is still to be done. This blog helps you to get started with sending logs to Panther, using syslog-ng. You can use either legacy syslog with TLS encryption (still a bit problematic) or the http() destination to send logs to the Panther HTTP API. https://www.syslog-ng.com/community/b/blog/posts/sending-logs-to-panther-us… Reducing the complexity of log management ----------------------------------------- It is easy to over-complicate log management. Almost all departments in a company need to log messages for their daily activities. However, installing several different log management and analysis systems in parallel is a nightmare both from a security and an operations perspective and wastes many resources. You cannot always reduce the number of log analysis systems, but you can reduce the complexity of log management. Let me show you, how. https://www.syslog-ng.com/community/b/blog/posts/reducing-the-complexity-of… Sending logs to Humio using the elasticsearch-http() destination of syslog-ng ------------- One of the most popular syslog-ng destinations is Elasticsearch. Humio, a log management provider, supports a broad range of ingest options and interfaces, including an Elasticsearch-compatible API. Last week, Humio announced Humio Community Edition, which provides the full Humio experience for free, with some limitations on daily ingestion and retention time. I tested the Community Edition, and it works perfectly well with syslog-ng. If you come from the Humio side, you might wonder what syslog-ng is. It is an application for high performance central log collection. Traditionally, syslog messages were collected centrally and saved to text files. Nowadays, syslog-ng acts more like a log management layer: collects log messages from hosts, saves them for long term storage, but also forwards them to multiple destinations, like SIEMs and other log analysis solutions. This way, it is enough to collect log messages only once, and syslog-ng delivers the right log messages to the right destinations in the right format, after some initial processing. Humio is available as a cloud service or self-hosted, where you can send all your logs for storage and analysis. It has an easy-to-use interface to query log messages which can be extended with further analytics possibilities from the Humio marketplace. >From this blog, you can learn how to get started with Humio and syslog-ng. While Humio provides many other APIs for log ingestion, I focus on the elasticsearch-http() destination of syslog-ng, demonstrating that there is no vendor lock-in: the same driver works equally well for Elastic’s Elasticsearch, AWS’s OpenSearch and for Humio. https://www.syslog-ng.com/community/b/blog/posts/sending-logs-to-humio-usin… WEBINARS * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

syslog-ng no longer receiving kernel messages
by John Covici 13 Dec '21

13 Dec '21

Hi folks. I am having a problem that syslog-ng no longer gets any kernel messages. It seems to get other messages, but not kernel messages. Here is what happened. In an update from kernel 5.10.70 to 5.10.82, I enabled the kernel race condition sanitizer. I looked at its output and decided that it would not do me any good, so I took that out and recompiled the kernel. However after rebooting the system, syslog-ng only gives me a few messages from when the kernel sanitizer was active in my /var/log/messages and the same for /var/log/kernel. I am using systemd, so I have the jernal and it is OK, all messages are posted correctly there. Here is my /etc/syslog-ng/syslog-ng.conf . @version: 3.34 # $Header: /var/cvsroot/gentoo-x86/app-admin/syslog-ng/files/3.4/syslog-ng.conf.gentoo,v 1.2 2013/06/02 01:18:35 mr_bones_ Exp $ # # Syslog-ng default configuration file for Gentoo Linux # https://bugs.gentoo.org/show_bug.cgi?id=426814 @include "scl.conf" options { dir_perm(0755); perm(0644); chain_hostnames(no); threaded(yes); chain_hostnames(no); # The default action of syslog-ng is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # how many messages syslog-ng missed (0). stats_freq(43200); # The default action of syslog-ng is to log a MARK line # to the file every 20 minutes. That's seems high for most # people so turn it down to once an hour. Set it to zero # if you don't want the functionality at all. mark_freq(0); keep_hostname(yes); }; source local { system() ; internal(); }; # *.emerg * filter f_9 { level(emerg); }; destination d_6 { usertty("*"); }; log { source(local); filter(f_9); destination(d_6); }; # kern.=debug /var/log/kernel filter f_1 { facility(kern) and level(debug..emerg); }; destination d_1 { file("/var/log/kernel" create_dirs(yes)); }; log { source(local); filter(f_1); destination(d_1); }; # kern.err /dev/console filter f_2 { facility(kern) and level(err..emerg); }; destination d_2 { file("/dev/console" create_dirs(yes)); }; log { source(local); filter(f_2); destination(d_2); }; # *.debug;mail.none;news.none;authpriv.none /var/log/messages filter f_3 { level(debug..emerg); }; filter f_4 { not facility(mail); }; filter f_5 { not facility(news); }; filter f_6 { not facility(authpriv); }; destination d_3 { file("/var/log/messages" create_dirs(yes)); }; log { source(local); filter(f_3); filter(f_4); filter(f_5); filter(f_6); destination(d_3); flags(final flow-control); }; # authpriv.* /var/log/secure filter f_7 { facility(authpriv) and level(debug..emerg); }; destination d_4 { file("/var/log/secure" create_dirs(yes)); }; log { source(local); filter(f_7); destination(d_4); }; # mail.* /var/log/maillog filter f_8 { facility(mail) and level(debug..emerg); }; destination d_5 { file("/var/log/maillog" create_dirs(yes)); }; log { source(local); filter(f_8); destination(d_5); }; # news.=crit /var/log/news/news.crit filter f_10 { facility(news) and level(crit..emerg); }; destination d_7 { file("/var/log/news/news.crit" create_dirs(yes)); }; log { source(local); filter(f_10); destination(d_7); }; # news.=err /var/log/news/news.err filter f_11 { facility(news) and level(err..emerg); }; destination d_8 { file("/var/log/news/news.err" create_dirs(yes)); }; log { source(local); filter(f_11); destination(d_8); }; # news.notice /var/log/news/news.notice filter f_12 { facility(news) and level(notice..emerg); }; destination d_9 { file("/var/log/news/news.notice" create_dirs(yes)); }; log { source(local); filter(f_12); destination(d_9); }; # local7.* /var/log/boot.log filter f_13 { facility(local7) and level(debug..emerg); }; destination d_10 { file("/var/log/boot.log" create_dirs(yes)); }; log { source(local); filter(f_13); destination(d_10); }; # local0.* /var/log/dhcpcd.log filter f_14 { facility(local0) and level(debug..emerg); }; So, what have I been doing wrong, or is it something else? Thanks in advance for any suggestions. destination d_11 { file("/var/log/dhcpcd.log" create_dirs(yes)); }; log { source(local); filter(f_14); destination(d_11); }; -- Your life is like a penny. You're going to lose it. The question is: How do you spend it? John Covici wb2una covici(a)ccs.covici.com

2 4

[patch] new feature for file destination: persistent symlinks
by a.biardi＠tiscali.it 07 Dec '21

07 Dec '21

Hi all, I'm new here. I love syslog-ng: it's well written and documented and each time I need some new feature, I dig up the documentation and I find it's already implemented! Like many others, I use macros in file destinations (say, "/var/log/cron.${YEAR}${MONTH}"), but I also want to maintain a persistent symlink to the current logfile (that would be something like "/var/log/cron") in order to have "easier" access if I'm grepping for something manually. I couldn't find this feature in syslog-ng, so I wrote the code myself; in the configuration file, it looks like this: destination d_file_cron { file("/var/log/cron.${YEAR}${MONTH}" symlink_as("/var/log/cron")); }; It works for me on my Slackware boxes. I'm sharing the patch I put together, hoping that it may be useful to others and perhaps even considered for inclusion in mainstream. From a functional perspective, the "symlink_as" file inherits both "create-dirs" and file ownership from its file destination (permissions are not applicable to symlinks, at least on linux). Patch applies cleanly to latest production release (3.33). I tried to keep the original coding style for easier review (admittedly, the code could have gone into the function that opens the file destination, but I didn't want to change too much). Thanks for the good work and happy logging! Andrea.

3 4

Re: [syslog-ng] [patch] new feature for file destination: persistent symlinks
by a.biardi＠tiscali.it 06 Dec '21

06 Dec '21

On Mon, 6 Dec 2021 15:28:18 +0000, "Gabor Nagy (gnagy)" <Gabor.Nagy(a)oneidentity.com> wrote: > Dear Andrea! > > I'm forwarding my answer to make sure you get it anyway. > Looking forward to hearing from you! > > Best Regards, > Gábor > ________________________________ > From: Gabor Nagy (gnagy) <Gabor.Nagy(a)oneidentity.com> > Sent: Sunday, December 5, 2021 17:49 > To: Syslog-ng users' and developers' mailing list > <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] [patch] new > feature for file destination: persistent symlinks > > Hello Andrea! > > Thank you for your contribution! We always welcome feedback and > contribution from the community! > > I've quickly checked your patch, and it looks good. Maybe minor > things could be changed, but we will see this during a thorough > review. > > We integrate patches via pull requests on GitHub. [1] > How would you like to continue this contribution? > Would you like to open a pull request on GitHub? > > I see a corner case of the feature: if the filename template is not > time-based, like e.g. $PROGRAM or $HOST, it can happen that the > symlink is changed several times in a short time. This is not an > argument against the feature, it's just a behaviour that should be > considered and documented. As I see, the feature in general creates a > symlink with the name given in "symlink_as" option when a new file is > opened. In other words: the configured symlink always points to the > latest file that were opened. > > > Best Regards, > Gábor > > [1] https://github.com/syslog-ng/syslog-ng > > ________________________________ > From: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu> on behalf of > a.biardi(a)tiscali.it <a.biardi(a)tiscali.it> Sent: Saturday, December 4, > 2021 15:29 To: syslog-ng(a)lists.balabit.hu <syslog-ng(a)lists.balabit.hu> > Subject: [syslog-ng] [patch] new feature for file destination: > persistent symlinks > > CAUTION: This email originated from outside of the organization. Do > not follow guidance, click links, or open attachments unless you > recognize the sender and know the content is safe. > > > Hi all, > > I'm new here. I love syslog-ng: it's well written and documented and > each time I need some new feature, I dig up the documentation and I > find it's already implemented! > > Like many others, I use macros in file destinations (say, > "/var/log/cron.${YEAR}${MONTH}"), but I also want to maintain a > persistent symlink to the current logfile (that would be > something like "/var/log/cron") in order to have "easier" access if > I'm grepping for something manually. > > I couldn't find this feature in syslog-ng, so I wrote the code myself; > in the configuration file, it looks like this: > > destination d_file_cron { > file("/var/log/cron.${YEAR}${MONTH}" symlink_as("/var/log/cron")); > }; > > It works for me on my Slackware boxes. I'm sharing the patch I put > together, hoping that it may be useful to others and perhaps even > considered for inclusion in mainstream. > > From a functional perspective, the "symlink_as" file inherits both > "create-dirs" and file ownership from its file destination > (permissions are not applicable to symlinks, at least on linux). > > Patch applies cleanly to latest production release (3.33). I tried to > keep the original coding style for easier review (admittedly, the code > could have gone into the function that opens the file destination, but > I didn't want to change too much). > > Thanks for the good work and happy logging! > > Andrea. Hi Gábor, Sorry for the delay, daytime job keeps me busy :) Will submit a PR through github asap. I did not consider the case where the template contains something like $PROGRAM or $HOST. As you point out, the code wouldn't fail but you'd get an ever-changing symlink of dubious usefulness. Worth documenting. Cheers, Andrea.

1 0

Selective application of keep-hostname()?
by Steve Bernacki 29 Nov '21

29 Nov '21

I have a use case where I need to selectively apply keep-hostname(yes) to messages from certain source IPs, while defaulting to keep-hostname(no) for the rest of my sources. I know I can apply this option on a per-source basis, but in this case I want to be able to selectively apply this option based on the source IP of the message. I'm trying to avoid having to set up an alternate port if I can. Thanks Steve

2 1

Insider 2021-11: 3.35; MacOS; mqtt() destination updates;
by Peter Czanik (pczanik) 24 Nov '21

24 Nov '21

Dear syslog-ng users, This is the 96th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS syslog-ng 3.35.1 available -------------------------- Version 3.35.1 of syslog-ng is released. New features include: - APT repo for x86 Debian and Ubuntu - Throttle filter - MQTT source - PROXY header before TLS payload You can read more details and download the syslog-ng source code at https://github.com/syslog-ng/syslog-ng/releases/tag/syslog-ng-3.35.1 Sending logs from syslog-ng store box to Splunk ----------------------------------------------- One of the most popular applications to feed Splunk with syslog messages is syslog-ng. However not everyone is happy to work on the command line anymore. This is where syslog-ng store box (SSB), an appliance built around syslog-ng, can help. The SSB GUI provides you not only with an easyto-use interface to configure most syslog-ng features, but also a search interface and complete log life cycle management. It can forward log messages to several destinations, recently also to Splunk’s HTTP Event Collector (HEC). From this blog you can learn about how SSB fits into your logging infrastructure and how to configure SSB for Splunk. https://www.syslog-ng.com/community/b/blog/posts/sending-logs-from-syslog-n… MacOS support ------------- For the past couple of months, Yash Mathne has been working on testing syslog-ng on MacOS as a GSoC (Google Summer of Code) student. He worked both on x86 and on the freshly released ARM hardware. And we have some good news here to share: while there is still room for improvement, most of syslog-ng works perfectly well on MacOS. You can read the two parts of his report at: Overview: https://www.syslog-ng.com/community/b/blog/posts/gsoc-report-syslog-ng-maco… Technical details: https://www.syslog-ng.com/community/b/blog/posts/status-of-syslog-ng-on-mac… Syslog-ng 3.34: MQTT destination with TLS and WebSocket support --------------------------------------------------------------- Version 3.33 of syslog-ng arrived with basic MQTT support. Version 3.34 has added many important features to it: user authentication, TLS support and WebSocket support. These features give you both security and flexibility while sending log messages to an MQTT broker. This blog helps you to make your first steps securing your MQTT connection. https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-3-34-mqtt-destin… WEBINARS * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

syslog-ng-stable rpm repos
by Peter Czanik (pczanik) 22 Nov '21

22 Nov '21

Hi, I just updated my syslog-ng-stable rpm repos to syslog-ng version 3.35. It was released a week ago, quite a few people already use it and nobody reported any problems. So, it should be a safe update. For new features and changes of 3.35, check: https://github.com/syslog-ng/syslog-ng/releases/tag/syslog-ng-3.35.1 For an overview of my unofficial RPM repositories for openSUSE / SLES, Fedora / RHEL (and compatibles), check: https://www.syslog-ng.com/community/b/blog/posts/overview-of-syslog-ng-rpm-… Bye, CzP Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 1