- syslog-ng - lists.balabit.hu

we need to know if Syslog supports any of these virtualization solution - Harvester, CaaS or HyperV. If not, what other virtualization solution is supported by Syslog.
by Mayekar, PrachiX 24 Apr '24

24 Apr '24

We need to know which virtualization solution is supported by syslog - Harvester CaaS or HyperV ? Thanks & Regards, Prachi Mayekar ITI-Network Services A Contingent Worker at Intel For assistance, please visit us at https://it.intel.com<https://it.intel.com/>

3 2

syslog-ng 4.7.1
by László Várady 19 Apr '24

19 Apr '24

Dear syslog-ng users, We are pleased to announce the 4.7.1 version of syslog-ng, which has been released and is now available on GitHub: https://github.com/syslog-ng/syslog-ng/releases/tag/syslog-ng-4.7.1 Packages <https://github.com/syslog-ng/syslog-ng#installation-from-binaries> are also available for various platforms. The AxoSyslog project provides cloud-ready container images <https://axoflow.com/cloud-ready-syslog-ng-images/>, Helm charts <https://axoflow.com/axosyslog-log-collection-for-kubernetes/>. 4.7.1 *This is the combination of the news entries of 4.7.0 and 4.7.1.* *4.7.1 hotfixed two crashes related to configuration reload.* Read Axoflow's blog post <https://axoflow.com/axosyslog-release-4-7/> for more details. You can read more about the new features in the AxoSyslog documentation <https://axoflow.com/docs/axosyslog-core/>. HighlightsCollecting Jellyfin logs The new jellyfin() source, reads Jellyfin logs from its log file output. Example minimal config: source s_jellyfin { jellyfin( base-dir("/path/to/my/jellyfin/root/log/dir") filename-pattern("log_*.log") ); }; For more details about Jellyfin logging, see: - https://jellyfin.org/docs/general/administration/configuration/#main-config… - https://jellyfin.org/docs/general/administration/configuration/#log-directo… As the jellyfin() source is based on a wildcard-file() source, all of the wildcard-file() source options are applicable, too. (#4802 <https://github.com/syslog-ng/syslog-ng/pull/4802>) Collecting *arr logs Use the newly added *arr() sources to read various *arr logs: - lidarr() - prowlarr() - radarr() - readarr() - sonarr() - whisparr() Example minimal config: source s_radarr { radarr( dir("/path/to/my/radarr/log/dir") ); }; The logging module is stored in the <prefix><module> name-value pair, for example: .radarr.module => ImportListSyncService. The prefix can be modified with the prefix() option. (#4803 <https://github.com/syslog-ng/syslog-ng/pull/4803>) Features - opentelemetry(), syslog-ng-otlp() source: Added concurrent-requests() option. This option configures the maximal number of in-flight gRPC requests per worker. Setting this value to the range of 10s or 100s is recommended when there are a high number of clients sending simultaneously. Ideally, workers() * concurrent-requests() should be greater or equal to the number of clients, but this can increase the memory usage. (#4827 <https://github.com/syslog-ng/syslog-ng/pull/4827>) - loki(): Support multi-tenancy with the new tenant-id() option (#4812 <https://github.com/syslog-ng/syslog-ng/pull/4812>) - s3(): Added support for authentication from environment. The access-key() and secret-key() options are now optional, which makes it possible to use authentication methods originated from the environment, e.g. AWS_... environment variables or credentials files from the ~/.aws/ directory. For more info, see: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.h… (#4881 <https://github.com/syslog-ng/syslog-ng/pull/4881>) - gRPC based drivers: Added channel-args() option. Affected drivers are: - bigquery() destination - loki() destination - opentelemetry() source and destination - syslog-ng-otlp() source and destination The channel-args() option accepts name-value pairs and sets channel arguments defined in https://grpc.github.io/grpc/core/group__grpc__arg__keys.html Example config: opentelemetry( channel-args( "grpc.loadreporting" => 1 "grpc.minimal_stack" => 0 ) ); (#4827 <https://github.com/syslog-ng/syslog-ng/pull/4827>) - ${TRANSPORT} macro: Added support for locally created logs. New values are: - "local+unix-stream" - "local+unix-dgram" - "local+file" - "local+pipe" - "local+program" - "local+devkmsg" - "local+journal" - "local+afstreams" - "local+openbsd" (#4777 <https://github.com/syslog-ng/syslog-ng/pull/4777>) - tags: Added new built-in tags that help identifying parse errors. New tags are: - "message.utf8_sanitized" - "message.parse_error" - "syslog.missing_pri" - "syslog.missing_timestamp" - "syslog.invalid_hostname" - "syslog.unexpected_framing" - "syslog.rfc3164_missing_header" - "syslog.rfc5424_unquoted_sdata_value" (#4804 <https://github.com/syslog-ng/syslog-ng/pull/4804>) - mqtt() source: Added ${MQTT_TOPIC} name-value pair. It is useful for the cases where topic() contains wildcards. Example config: log { source { mqtt(topic("#")); }; destination { stdout(template("${MQTT_TOPIC} - ${MESSAGE}\n")); }; }; (#4824 <https://github.com/syslog-ng/syslog-ng/pull/4824>) - template(): Added a new template function: $(tags-head) This template function accepts multiple tag names, and returns the first one that is set. Example config: # resolves to "bar" if "bar" tag is set, but "foo" is not template("$(tags-head foo bar baz)") (#4804 <https://github.com/syslog-ng/syslog-ng/pull/4804>) - s3(): Use default AWS URL if url() is not set. (#4813 <https://github.com/syslog-ng/syslog-ng/pull/4813>) - opentelemetry(), syslog-ng-otlp() source: Added log-fetch-limit() option. This option can be used to fine tune the performance. To minimize locking while moving messages between source and destination side queues, syslog-ng can move messages in batches. The log-fetch-limit() option sets the maximal size of the batch moved by a worker. By default it is equal to log-iw-size() / workers(). (#4827 <https://github.com/syslog-ng/syslog-ng/pull/4827>) - dqtool: add option for truncating (compacting) abandoned disk-buffers (#4875 <https://github.com/syslog-ng/syslog-ng/pull/4875>) Bugfixes - opentelemetry(): fix crash when an invalid configuration needs to be reverted (#4910 <https://github.com/syslog-ng/syslog-ng/pull/4910>) - gRPC drivers: fixed a crash when gRPC drivers were used and syslog-ng was reloaded (#4909 <https://github.com/syslog-ng/syslog-ng/pull/4909>) - opentelemetry(), syslog-ng-otlp() source: Fixed a crash. It occurred with multiple workers() during high load. (#4827 <https://github.com/syslog-ng/syslog-ng/pull/4827>) - rename(): Fixed a bug, which always converted the renamed NV pair to string type. (#4847 <https://github.com/syslog-ng/syslog-ng/pull/4847>) - With IPv6 disabled, there were linking errors (#4880 <https://github.com/syslog-ng/syslog-ng/pull/4880>) Metrics - http(): Added a new counter for HTTP requests. It is activated on stats(level(1));. Example metrics: syslogng_output_http_requests_total{url="http://localhost:8888/bar",response_code="200",driver="http",id="#anon-destination0#0"} 16 syslogng_output_http_requests_total{url="http://localhost:8888/bar",response_code="401",driver="http",id="#anon-destination0#0"} 2 syslogng_output_http_requests_total{url="http://localhost:8888/bar",response_code="502",driver="http",id="#anon-destination0#0"} 1 syslogng_output_http_requests_total{url="http://localhost:8888/foo",response_code="200",driver="http",id="#anon-destination0#0"} 24 (#4805 <https://github.com/syslog-ng/syslog-ng/pull/4805>) - gRPC based destination drivers: Added gRPC request related metrics. Affected drivers: - opentelemetry() - syslog-ng-otlp() - bigquery() - loki() Example metrics: syslogng_output_grpc_requests_total{driver="syslog-ng-otlp",url="localhost:12345",response_code="ok"} 49 syslogng_output_grpc_requests_total{driver="syslog-ng-otlp",url="localhost:12345",response_code="unavailable"} 11 (#4811 <https://github.com/syslog-ng/syslog-ng/pull/4811>) - New metric to monitor destination reachability syslogng_output_unreachable is a bool-like metric, which shows whether a destination is reachable or not. sum() can be used to count all unreachable outputs, hence the negated name. It is currently available for the network(), syslog(), unix-*() destinations, and threaded destinations (http(), opentelemetry(), redis() , mongodb(), python(), etc.). (#4876 <https://github.com/syslog-ng/syslog-ng/pull/4876>) - destinations: Added "syslogng_output_event_retries_total" counter. This counter is available for the following destination drivers: - amqp() - bigquery() - http() and all http based drivers - java() - kafka() - loki() - mongodb() - mqtt() - opentelemetry() - python() and all python based drivers - redis() - riemann() - smtp() - snmp() - sql() - stomp() - syslog-ng-otlp() Example metrics: syslogng_output_event_retries_total{driver="http",url="http://localhost:8888/${path}",id="#anon-destination0#0"} 5 (#4807 <https://github.com/syslog-ng/syslog-ng/pull/4807>) - syslogng_memory_queue_capacity Shows the capacity (maximum possible size) of each queue. Note that this metric publishes log-fifo-size(), which only limits non-flow-controlled messages. Messages coming from flow-controlled paths are not limited by log-fifo-size(), their corresponding source log-iw-size() is the upper limit. (#4831 <https://github.com/syslog-ng/syslog-ng/pull/4831>) Other changes - opentelemetry(), syslog-ng-otlp() source: Changed the backpressure behavior. syslog-ng no longer returns UNAVAILABLE to the gRPC request, when it cannot forward the received message because of backpressure. Instead, syslog-ng will block until the destination can accept more messages. (#4827 <https://github.com/syslog-ng/syslog-ng/pull/4827>) - opentelemetry(), syslog-ng-otlp() source: log-iw-size() is now split between workers. (#4827 <https://github.com/syslog-ng/syslog-ng/pull/4827>) - APT packages: Dropped Debian Buster support. Old packages are still available, but new syslog-ng versions will not be available on Debian Buster (#4840 <https://github.com/syslog-ng/syslog-ng/pull/4840>) - dbld: AlmaLinux 8 support (#4902 <https://github.com/syslog-ng/syslog-ng/pull/4902>) syslog-ng Discord For a bit more interactive discussion, join our Discord server: [image: Axoflow Discord Server] <https://discord.gg/E65kP9aZGm> Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Arpad Kunszt, Attila Szakacs, Balazs Scheidler, Bálint Horváth, Hofi, Kovács, Gergő Ferenc, László Várady, Peter Marko, shifter

1 0

we need to know if Syslog supports any of these virtualization solution - Harvester, CaaS or HyperV. If not, what other virtualization solution is supported by Syslog.
by Mayekar, PrachiX 16 Apr '24

16 Apr '24

Hi All, We are migrating the VMs from VMware to new solution (Harvester, CaaS, HyperV). Hence, we need to know if Syslog supports any of these virtualization solution - Harvester, CaaS or HyperV. If not, what other virtualization solution is supported by Syslog. Thanks & Regards, Prachi Mayekar ITI-Network Services A Contingent Worker at Intel For assistance, please visit us at https://it.intel.com<https://it.intel.com/>

3 2

The syslog-ng Insider 2024-04: PAM Essentials; XML Eventlog; multi-line logs
by Peter Czanik (pczanik) 11 Apr '24

11 Apr '24

Dear syslog-ng users, This is the 119th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Collecting One Identity Cloud PAM Essentials logs using syslog-ng ----------------------------------------------- One Identity Cloud PAM Essentials is the latest security product by One Identity. It provides asset management as well as secure and monitored remote access for One Identity Cloud users to hosts on their local network. I had a chance to test PAM Essentials while still in development. While there, I also integrated it with syslog-ng. >From this blog, you can learn what PAM Essentials is, and how you can collect its logs using syslog-ng. My next blog will show you how to work with the collected log messages and create alerts when somebody connects to a host on your local network using PAM Essentials. https://www.syslog-ng.com/community/b/blog/posts/collecting-one-identity-cl… Dedicated Windows XML eventlog parser in syslog-ng -------------------------------------------------- Version 4.6 of syslog-ng introduced windows-eventlog-xml-parser(), a dedicated parser for XML-formatted event logs from Windows. It makes the EventData portion of log messages more useful, as it combines two arrays into a list of name-value pairs. https://www.syslog-ng.com/community/b/blog/posts/dedicated-windows-xml-even… Working with multi-line logs in syslog-ng ----------------------------------------- Most log messages fit on a single line. However, Windows and some developer tools and services, like Tomcat, write multi-line log messages. These can come in various formats. For example, new log messages start with a date in a specific format. You use the multi-line-prefix() of the syslog-ng file() source to send multi-line messages as single messages instead of line by line. I must admit that I have never seen multi-line logs in production. I am not a developer, do not run Tomcat or Windows. However, recently I tested a software on Windows, which produced multi-line log messages. https://www.syslog-ng.com/community/b/blog/posts/working-with-multi-line-lo… WEBINARS * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

How does syslog-ng detect the availability of the remote end
by Grant Guo 03 Apr '24

03 Apr '24

Hi, Recently I am using syslog-ng’s docker container as syslog generator as the following: docker run -d \ --cpus=2 \ --name $name \ -v ${PWD}/syslog.log:/data/syslog.log \ --entrypoint="" \ --network=host \ balabit/syslog-ng:latest \ /usr/bin/loggen --dgram --inet --dont-parse --loop-reading --size 8192 --rate $rate --interval $duration --read-file /data/syslog.log $ip 514 I found something interesting, if my syslog server stopped, the client syslog-ng containers stopped too. So I would like to know if syslog-ng detected the availability of the remote end? If so, how? Besides, I didn’t see ICMP packets between the client and server using tcpdump. Thanks Grant *** Please note that this message and any attachments may contain confidential and proprietary material and information and are intended only for the use of the intended recipient(s). If you are not the intended recipient, you are hereby notified that any review, use, disclosure, dissemination, distribution or copying of this message and any attachments is strictly prohibited. If you have received this email in error, please immediately notify the sender and destroy this e-mail and any attachments and all copies, whether electronic or printed. Please also note that any views, opinions, conclusions or commitments expressed in this message are those of the individual sender and do not necessarily reflect the views of Fortinet, Inc., its affiliates, and emails are not binding on Fortinet and only a writing manually signed by Fortinet's General Counsel can be a binding commitment of Fortinet to Fortinet's customers or partners. Thank you. ***

1 0

The syslog-ng Insider 2024-03: MacOS; OpenTelemetry;
by Peter Czanik (pczanik) 14 Mar '24

14 Mar '24

Dear syslog-ng users, This is the 118th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Native MacOS source in syslog-ng -------------------------------- You know that support for MacOS is important when every third visitor at the syslog-ng booth of Red Hat Summit asks if syslog-ng works on MacOS. With the upcoming syslog-ng version 4.6.0, syslog-ng not only compiles on MacOS, but it also collects local log messages natively. >From this blog you can learn how to compile syslog-ng yourself, options of the MacOS source, and also a bit of history. https://www.syslog-ng.com/community/b/blog/posts/native-macos-source-in-sys… Using OpenTelemetry between syslog-ng instances ----------------------------------------------- Do you have to forward large amounts of logs between two syslog-ng instances? OTLP (OpenTelemetry protocol) support in syslog-ng can solve this problem. Just like the ewmm() destination, syslog-ng-otlp() forwards most name-value pairs, however, unlike a tcp() connection, it scales well with multiple CPU cores. Support for OpenTelemetry was added to syslog-ng a couple of releases ago. OpenTelemetry is an observability framework, mainly used in Linux / Cloud / Kubernetes environments. However, I already had users asking to make this feature available on FreeBSD. (It already worked once, but now it fails to compile again.) Version 4.6.0 added many new OTLP-related enhancements. Batching and multiple workers make OTLP connections significantly faster, while compression can save you bandwidth at the expense of some more CPU usage. This changes the syslog-ng-otlp() destination from an interesting experiment into something really useful. It enables you to send a lot more log messages between two syslog-ng instances than with a tcp() connection, while using less bandwidth. https://www.syslog-ng.com/community/b/blog/posts/using-opentelemetry-betwee… Collecting even more logs on MacOS using syslog-ng -------------------------------------------------- Version 4.6.0 of syslog-ng features not just a new, native log collector for MacOS, but also darwin-oslog-stream(), which can also collect non-persistent log events. Beware that it can collect many megabytes of logs even in just a few minutes! However, if you need to debug MacOS, these logs might come handy. https://www.syslog-ng.com/community/b/blog/posts/collecting-even-more-logs-… WEBINARS * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

syslog-ng service keeps restarting on systemd
by Darren Fuller 29 Feb '24

29 Feb '24

Hello, I have installed syslog-ng OSE from source on Amazon Linux 2023 distribution (as there is no EPEL available on that distribution nor are there any precompiled binaries for it) When i start up syslog-ng in the local terminal with syslog-ng -Fedv , it runs fine forever.. But when i start it with systemd : systemctl start syslog-ng, the command times out (but does start in the background) and then it the service restarts every two minutes or so. I am stumped. any thoughts? where have i gone wrong? appreciate any assistance you can offer. thanks, Darren More information: ------------ i compiled with the following and saw no errors: > ./configure --disable-smtp --disable-http --disable-python --disable-json > make > make install ------------ I created the systemd service file as so: > cat /etc/systemd/system/syslog-ng.service [Unit] Description=System Logger Daemon Documentation=man:syslog-ng(8) After=network.target [Service] Type=notify PIDFile=/run/syslogd.pid ExecStart=/usr/local/sbin/syslog-ng -F -p /run/syslogd.pid ExecReload=/bin/kill -HUP $MAINPID StandardOutput=null Restart=on-failure [Install] WantedBy=multi-user.target ------------ My config is here: ------------ > cat /usr/local/etc/syslog-ng.conf @include “scl.conf” source s_local { system(); internal(); }; destination d_local { file(“/var/log/messages”); file(“/var/log/messages-kv.log” template(“$ISODATE $HOST $(format-welf --scope all-nv-pairs)\n”) frac-digits(3)); }; log { source(s_local); destination(d_local); }; @include /etc/syslog-ng.d/*.conf > cat /etc/syslog.d/syslognet.conf options { long_hostnames(off); sync(0); keep_hostname(yes); owner(“splunk”); group(“splunk”); perm(0640); create_dirs(yes); dir_perm(0750); }; source s_tcp_net { tcp(ip(0.0.0.0) port(41514)); }; source s_tls_net { network ( ip(0.0.0.0) port(42514) transport(“tls”) tls( key-file(“/path/to/syslog-ng.key”) cert-file(“/path/to/syslog-ng.crt”) peer-verify(no) ) ); }; destination d_net { file(“/data/syslog/app/${SOURCEIP}/${YEAR}${MONTH}${DAY}-app-data.log”); }; log { source(s_tcp_net); source(s_tls_net); destination(d_net); }; ------------ Here is what i see when i start it with systemd: > systemctl start syslog-ng Job for syslog-ng.service failed because a timeout was exceeded. See “systemctl status syslog-ng.service” and “journalctl -xeu syslog-ng.service” for details. > systemctl status syslog-ng.service syslog-ng.service - System Logger Daemon Loaded: loaded (/etc/systemd/system/syslog-ng.service; enabled; preset: enabled) Active: activating (start) since Wed 2024-02-28 15:25:01 UTC; 35s ago Docs: man:syslog-ng(8) Main PID: 488557 (syslog-ng) Tasks: 4 (limit: 2322) Memory: 2.5M CPU: 39ms CGroup: /system.slice/syslog-ng.service └─488557 /usr/local/sbin/syslog-ng -F -p /run/syslogd.pid Feb 28 15:25:01 ip-172-31-7-110.ca-central-1.compute.internal systemd[1]: Starting syslog-ng.service - System Logger Daemon... ------------ And here is what's in /var/log/messages showing the restart: > cat /var/log/messages | grep syslog-ng Feb 28 14:38:24 ip-172-31-7-110 syslog-ng[447117]: syslog-ng shutting down; version=‘4.6.0’ Feb 28 14:38:25 ip-172-31-7-110 syslog-ng[447177]: syslog-ng starting up; version=‘4.6.0’ Feb 28 14:38:26 ip-172-31-7-110 syslog-ng[447177]: Syslog connection accepted; fd=‘26’, client=‘AF_INET(1.2.3.4:27717)’, local=‘AF_INET(0.0.0.0:41514)’ Feb 28 14:38:26 ip-172-31-7-110 syslog-ng[447177]: Syslog connection accepted; fd=‘27’, client=‘AF_INET(1.2.3.4:2300)’, local=‘AF_INET(0.0.0.0:41514)’ Feb 28 14:38:26 ip-172-31-7-110 syslog-ng[447177]: Syslog connection accepted; fd=‘28’, client=‘AF_INET(1.2.3.4:31585)’, local=‘AF_INET(0.0.0.0:41514)’ Feb 28 14:38:26 ip-172-31-7-110 syslog-ng[447177]: Syslog connection accepted; fd=‘29’, client=‘AF_INET(1.2.3.4:17761)’, local=‘AF_INET(0.0.0.0:41514)’ Feb 28 14:39:55 ip-172-31-7-110 syslog-ng[447177]: syslog-ng shutting down; version=‘4.6.0’ Feb 28 14:39:55 ip-172-31-7-110 syslog-ng[447234]: syslog-ng starting up; version=‘4.6.0’ Feb 28 14:39:56 ip-172-31-7-110 syslog-ng[447234]: Syslog connection accepted; fd=‘25’, client=‘AF_INET(1.2.3.4:7213)’, local=‘AF_INET(0.0.0.0:41514)’ Feb 28 14:39:56 ip-172-31-7-110 syslog-ng[447234]: Syslog connection accepted; fd=‘26’, client=‘AF_INET(1.2.3.4:9268)’, local=‘AF_INET(0.0.0.0:41514)’ Feb 28 14:39:56 ip-172-31-7-110 syslog-ng[447234]: Syslog connection accepted; fd=‘27’, client=‘AF_INET(1.2.3.4:13128)’, local=‘AF_INET(0.0.0.0:41514)’ Feb 28 14:39:56 ip-172-31-7-110 syslog-ng[447234]: Syslog connection accepted; fd=‘28’, client=‘AF_INET(1.2.3.4:3928)’, local=‘AF_INET(0.0.0.0:41514)’ Feb 28 14:41:25 ip-172-31-7-110 syslog-ng[447234]: syslog-ng shutting down; version=‘4.6.0’ Feb 28 14:41:26 ip-172-31-7-110 syslog-ng[447385]: syslog-ng starting up; version=‘4.6.0’ Feb 28 14:41:27 ip-172-31-7-110 syslog-ng[447385]: Syslog connection accepted; fd=‘25’, client=‘AF_INET(1.2.3.4:1382)’, local=‘AF_INET(0.0.0.0:41514)’ Feb 28 14:41:27 ip-172-31-7-110 syslog-ng[447385]: Syslog connection accepted; fd=‘26’, client=‘AF_INET(1.2.3.4:5430)’, local=‘AF_INET(0.0.0.0:41514)’ Feb 28 14:41:27 ip-172-31-7-110 syslog-ng[447385]: Syslog connection accepted; fd=‘27’, client=‘AF_INET(1.2.3.4:6373)’, local=‘AF_INET(0.0.0.0:41514)’ Feb 28 14:41:27 ip-172-31-7-110 syslog-ng[447385]: Syslog connection accepted; fd=‘28’, client=‘AF_INET(1.2.3.4:26287)’, local=‘AF_INET(0.0.0.0:41514)’ Feb 28 14:42:56 ip-172-31-7-110 syslog-ng[447385]: syslog-ng shutting down; version=‘4.6.0’ Feb 28 14:42:56 ip-172-31-7-110 syslog-ng[447443]: syslog-ng starting up; version=‘4.6.0’ Feb 28 14:42:57 ip-172-31-7-110 syslog-ng[447443]: Syslog connection accepted; fd=‘25’, client=‘AF_INET(1.2.3.4:15165)’, local=‘AF_INET(0.0.0.0:41514)’ Feb 28 14:42:57 ip-172-31-7-110 syslog-ng[447443]: Syslog connection accepted; fd=‘26’, client=‘AF_INET(1.2.3.4:6967)’, local=‘AF_INET(0.0.0.0:41514)’ Feb 28 14:42:57 ip-172-31-7-110 syslog-ng[447443]: Syslog connection accepted; fd=‘27’, client=‘AF_INET(1.2.3.4:27444)’, local=‘AF_INET(0.0.0.0:41514)’ Feb 28 14:42:57 ip-172-31-7-110 syslog-ng[447443]: Syslog connection accepted; fd=‘28’, client=‘AF_INET(1.2.3.4:20758)’, local=‘AF_INET(0.0.0.0:41514)’

3 2

running RHEL 9
by Linn, William 19 Feb '24

19 Feb '24

Has anyone seen any issues with syslog-ng running on RHEL 9? Any pointers, caveats? Thanks William ---------------------------------------------------------------------- This message, and any attachment(s), is for the intended recipient(s) only, may contain information that is privileged, confidential and/or proprietary and subject to important terms and conditions available at http://www.bankofamerica.com/electronic-disclaimer. If you are not the intended recipient, please delete this message. For more information about how Bank of America protects your privacy, including specific rights that may apply, please visit the following pages: https://business.bofa.com/en-us/content/global-privacy-notices.html (which includes global privacy notices) and https://www.bankofamerica.com/security-center/privacy-overview/ (which includes US State specific privacy notices such as the http://www.bankofamerica.com/ccpa-notice)

3 3

The syslog-ng Insider 2024-02: OpenObserve; configuration check; build services;
by Peter Czanik (pczanik) 15 Feb '24

15 Feb '24

Dear syslog-ng users, This is the 117th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Version 4.5.0 of syslog-ng is now available with OpenObserve JSON API support ------------------------------------------------ Recently, syslog-ng 4.5.0 was released with many new features. These include sending logs to OpenObserve using its JSON API, support for Google Pub/Sub, a new macro describing message transport mechanisms like RFC 3164 + TCP, an SSL option to ignore validity periods, and many more. You can find a full list of new features and bug fixes in the release notes at: https://github.com/syslog-ng/syslog-ng/releases/tag/syslog-ng-4.5.0 In this blog, you can find some pointers on how to install the very latest syslog-ng version and learn how you can configure syslog-ng to use the OpenObserve JSON API. https://www.syslog-ng.com/community/b/blog/posts/version-4-5-0-of-syslog-ng… Syslog-ng PE can now send logs to Google BigQuery ------------------------------------------------- Version 7.0.34 of syslog-ng PE can now send logs to Google BigQuery destinations. You can learn more about how it works from the syslog-ng PE documentation: https://support.oneidentity.com/technical-documents/syslog-ng-premium-editi… Syslog-ng can now do a full configuration check ----------------------------------------------- One of the most frequent syslog-ng feature requests is now resolved. Welcome the --check-startup option, allowing you to check the syntax and also spot spelling mistakes! https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-can-now-do-a-ful… How build services make life easier for upstream developers ----------------------------------------------------------- Many Linux distributions provide build services under various names: openSUSE Build Service (OBS), Fedora Copr, and so on. These resources are indispensable for upstream developers, and also for their users. I will demonstrate this through some examples from the syslog-ng project. Note: this blog is loosely based on a talk idea I had for the FOSDEM Distributions Devroom. There is no deep technical information about syslog-ng in this blog. This is more like a history of syslog-ng packaging, and how the fantastic tools by openSUSE and Fedora made it a lot easier and made me an active part of these communities. https://www.syslog-ng.com/community/b/blog/posts/how-build-services-make-li… WEBINARS * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

The syslog-ng Insider 2024-01: HTTP; Cloudflare; systemd-journal; Humio / Logscale;
by Peter Czanik (pczanik) 18 Jan '24

18 Jan '24

Dear syslog-ng users, This is the 116th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Why use a http()-based destination in syslog-ng? ------------------------------------------------ Logging is not just syslog anymore. Still, many syslog-ng users stick to using one of the syslog protocols for log transport and flat files for log storage. While most SIEMs and log analytics tools can receive syslog messages or read them using their own agents, in most cases, you can use the http() destination of syslog-ng as well to send logs to them. You gain extreme performance and an architecture that is easier to maintain. Some of the drivers in syslog-ng built on top of the http() destination include elasticsearch-http() destination for sending logs to Elasticsearch / OpenSearch. Many other services utilizing the Elasticsearch Bulk API exist, and there are also others using Sumo Logic, Splunk, and so on. Of course, there are some destinations where performance is not a major concern (or can even be a drawback, in fact). For example,various instant messaging services used for alerting (like Telegram or Slack) belong in this category. You can also read the API docs and write a new destination based on http() yourself. https://www.syslog-ng.com/community/b/blog/posts/why-use-a-http--based-dest… An overview of Cloudflare's logging pipeline -------------------------------------------- One of the roles of Cloudflare's Observability Platform team is managing the operation, improvement, and maintenance of our internal logging pipelines. These pipelines are used to ship debugging logs from every service across Cloudflare’s infrastructure into a centralised location, allowing our engineers to operate and debug their services in near real time. In this post, we’re going to go over what that looks like, how we achieve high availability, and how we meet our Service Level Objectives (SLOs) while shipping close to a million log lines per second. https://blog.cloudflare.com/an-overview-of-cloudflares-logging-pipeline/ Working with multiple systemd-journal namespaces in syslog-ng ------------------------------------------------------------- Initial support for systemd-journal namespaces is available in syslog-ng 3.29. However, only version 4.4.0 allows you to work with multiple namespaces in your syslog-ng configuration. So, what changed in the latest version of syslog-ng? Previously, you could only configure a single systemd-journal() source in syslog-ng. By default, it collected logs from all namespaces, but you could configure it to collect log messages from a single one exclusively. This means that logs from other namespaces could not be collected by syslog-ng. Version 4.4.0 allows you to use multiple systemd-journal() source drivers in the configuration, as long as each source uses a unique namespace. https://www.syslog-ng.com/community/b/blog/posts/working-with-multiple-syst… Logging to Humio / Logscale simplified in syslog-ng --------------------------------------------------- Logging into Humio (which was recently re-branded to Falcon LogScale) was available for years, using their Elasticsearch compatible API. However, according to Humio developers, it is slightly slower than other APIs for log ingestion. Axoflow contributed a Logscale destination to syslog-ng, which uses Logscale’s native API. I did not measure if there is really a performance difference, however it is definitely easier to configure it. WEBINARS * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0