- syslog-ng - lists.balabit.hu

syslog-ng 3.38 UDP errors and missing messages
by Taavi Ansper 11 Feb '25

11 Feb '25

Hi I am running a syslog server that is having problems getting all of the data via UDP. I have gone through most of the tips for making the performance better, but something is still amiss. 1. The syslog-ng server is running in an VM. 8vcpu and 8gb RAM. Debian 12. 2. After restarting or starting the server the UDP errors are tamed. nstat -asz | grep Error UdpInErrors 0 0.0 UdpRcvbufErrors 0 0.0 After 15 minutes or so the errors start to appear. (This amount of errors is after some time.) UdpInErrors 6734175 0.0 UdpRcvbufErrors 6734175 0.0 After some time it starts to receive the packets again. I can see all of this clearly in my graylog graphics that sucks that log data in. (added picture) https://imgur.com/a/eeRpovh It goes like a sinewave up and down every 15 minutes or so. My config looks like this. options { ts-format(iso); frac-digits(6); keep-timestamp(no); chain-hostnames(no); keep-hostname(no); use-dns(yes); use-fqdn(yes); create-dirs(yes); dir-owner(root); dir-group(adm); dir-perm(0750); owner(root); group(adm); perm(0640); log-fifo-size(100000); }; source clients { network( transport(udp) so-rcvbuf(32MiB) log-iw-size(250k) log-fetch-limit(10k) ); network( transport(tcp) max-connections(100) so-rcvbuf(32MiB) log-iw-size(250k) log-fetch-limit(10k) ); }; source forward { network( port(6514) transport(tcp) keep-timestamp(yes) keep-hostname(yes) ); }; destination file { file("/srv/log/${HOST}/${FACILITY}.log"); }; destination forward { network( "192.168.X.X" port(6514) transport(tcp) so-keepalive(yes) disk-buffer( mem-buf-size(512M) disk-buf-size(2048M) reliable(yes) ) ); }; destination graylog { network( "192.168.X.X" port(5140) flags(syslog-protocol) transport(tcp) so-keepalive(yes) disk-buffer( mem-buf-size(512M) disk-buf-size(2048M) reliable(yes) ) ); }; filter graylog { host("^gw-example-") }; log { source(clients); destination(file); destination(forward); flags(flow-control); }; log { source(forward); destination(file); flags(flow-control); }; log { source(clients); filter(graylog); destination(graylog); flags(flow-control); }; My rmem_max is set like so: sysctl net.core.rmem_max net.core.rmem_max = 33445532 Any suggestions? Best regards -- Taavi Ansper taavi.ansper(a)cyber.ee

2 1

The syslog-ng Insider 2025-01: Alpine Linux; Leap 16.0; Alma Linux
by Peter Czanik (pczanik) 16 Jan '25

16 Jan '25

Dear syslog-ng users, This is the 127th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS A syslog-ng container image based on Alpine Linux ------------------------------------------------- Recently, someone suggested I should check out Alpine Linux and prepare a syslog-ng container image based on it. While not supported by the syslog-ng project, an Alpine-based syslog-ng container image already exist as part of the Linuxserver project. https://www.syslog-ng.com/community/b/blog/posts/a-syslog-ng-container-imag… Call for testing: syslog-ng in openSUSE Leap 16.0 ------------------------------------------------- Last week, I submitted syslog-ng to openSUSE Leap 16.0. While the distro is still in a pre-alpha stage, everything already works for me as expected. Well, except for syslog-ng, where I found a number of smaller problems. As such, this blog is a call for testing, both for syslog-ng on openSUSE Leap 16.0 and also for the distribution itself.https://www.syslog-ng.com/community/b/blog/posts/call-for-testing-sy… Experimental syslog-ng container image based on Alma Linux ---------------------------------------------------------- The official syslog-ng container image is based on Debian Stable. However, we’ve been getting requests for an RPM-based image for many years. So, I made an initial version available based on Alma Linux and now I need your feedback about it! https://www.syslog-ng.com/community/b/blog/posts/experimental-syslog-ng-con… WEBINARS * New webinar: “Performance tuning for syslog-ng deployments”. You can register for it at https://www.syslog-ng.com/event/performance-tuning-for-syslog-ng-deployment… * You can learn about upcoming webinars and browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Creation of manual pages
by Generic Secure-Logging 03 Jan '25

03 Jan '25

syslog-ng features manual pages for the Unix manual system in the doc/ subdirectory. However, these pages are not created when configuring, building and installing syslog-ng with CMake. The autotools config system creates a Makefile with a target manpages when calling configure with the option --enable-manpages. However, the autotools build is broken. The call of make manpages results in the following error ./configure: line 7001: syntax error near unexpected token `newline' ./configure: line 7001: `yes:' make: *** [Makefile:13726: config.status] Error 2 Does anyone know how to create the manual pages in the current version? The information in this e-mail is confidential. The contents may not be disclosed or used by anyone other than the addressee. Access to this e-mail by anyone else is unauthorised. If you are not the intended recipient, please notify Airbus immediately and delete this e-mail. Airbus cannot accept any responsibility for the accuracy or completeness of this e-mail as it has been sent over public networks. If you have any concerns over the content of this message or its Accuracy or Integrity, please contact Airbus immediately. All outgoing e-mails from Airbus are checked using regularly updated virus scanning software but you should take whatever measures you deem to be appropriate to ensure that this message and any attachments are virus free.

1 0

Syslog server chaining issue
by Maurya, Shivani 12 Dec '24

12 Dec '24

Hi All, I am using 2 syslog servers on version 3.31. The devices are sending syslog message to 1st syslog server. The 1st syslog server is forwarding the same message to 2nd syslog server. Device --> Syslog Server 1 --> Syslog Server 2 The configurations of 1st syslog server - Options - keep-hostname(yes); use-dns(yes); use-fqdn(yes); Destination - destination d_sec { udp("IP_of_second_syslog" port(514) template("${ISODATE} ${HOST} ${PRIORITY} ${MSG}\n") template-escape(no)); }; The configuration of 2nd syslog server - destination d_syslogFile { file("/var/log/syslog.log" template("${R_ISODATE} ${HOST} ${PRIORITY} ${FACILITY} ${PROGRAM} ${MSG}\n") template-escape(no)); }; Problem - When the syslog message is getting logged at 2nd Syslog server, the ${PRIORITY} of the message is always "notice". And the original severity/priotity of the message is getting captured in the ${PROGRAM} macro. How to capture the priority of the forwarded message on 2nd Syslog server in the ${PRIORITY} macro instead of ${PROGRAM} macro? Regards, Shivani Maurya

3 3

The syslog-ng Insider 2024-12: FreeBSD audit; 4.8.1; conferences
by Peter Czanik (pczanik) 05 Dec '24

05 Dec '24

Dear syslog-ng users, This is the 126th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS FreeBSD audit source for syslog-ng ---------------------------------- Two weeks ago, I was at EuroBSDcon and received a feature request for syslog-ng. The user wanted to collect FreeBSD audit logs together with other logs using syslog-ng. Writing a native driver in C is time consuming. However, creating an integration based on the program() source of syslog-ng is not that difficult. This blog shows you the current state of the FreeBSD audit source, how it works, and its limitations. It is also a request for feedback. https://www.syslog-ng.com/community/b/blog/posts/freebsd-audit-source-for-s… Version 4.8.1 of syslog-ng is now available ------------------------------------------- Version 4.8.1 was released recently. As you could guess from the version number change, it is primarily a bug fix release, but some minor features also slipped in. From this blog, you can learn what changed in syslog-ng 4.8.1 and where you can get its latest stable version. https://www.syslog-ng.com/community/b/blog/posts/version-4-8-1-of-syslog-ng… Where should I present syslog-ng and sudo? ------------------------------------------ Recently I was asked the same question both at my workplace and at EuroBSDCon, the conference where I was presenting: where do you talk next? I had no definite answer. Of course, I am looking forward to the FOSDEM CfP, but I am also looking for new conferences to present syslog-ng and sudo. Do you have any recommendations? https://www.syslog-ng.com/community/b/blog/posts/where-should-i-present-sys… WEBINARS * New webinar: “Performance tuning for syslog-ng deployments”. You can register for it at https://www.syslog-ng.com/event/performance-tuning-for-syslog-ng-deployment… * You can learn about upcoming webinars and browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

The syslog-ng Insider 2024-11: testing; Quickwit; MacPorts
by Peter Czanik (pczanik) 14 Nov '24

14 Nov '24

Dear syslog-ng users, This is the 125th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS A call for syslog-ng testing ---------------------------- After the last syslog-ng release, we started a campaign to close open issues on GitHub. We'd like to continue this effort and call for collaboration from our users and contributors to make OSE even more stable. While unit tests are great (and we do many tests in-house), nothing can replace using syslog-ng in real-world situations. This blog collects some resources about how you can start testing the latest syslog-ng release from GitHub. https://www.syslog-ng.com/community/b/blog/posts/a-call-for-syslog-ng-testi… Working with Quickwit --------------------- We are always looking for new ways to store log messages. Quickwit is a new contender, designed for log storage, and among others, it also provides an Elasticsearch-compatible API. From this blog, you can learn about Quickwit, and how to forward log messages from syslog-ng to it using the Elasticsearch-compatible API. https://www.syslog-ng.com/community/b/blog/posts/first-steps-with-quickwit-… You can also use the OpenTelemetry protocol to send logs to Quickwit with syslog-ng. https://www.syslog-ng.com/community/b/blog/posts/sending-logs-to-quickwit-u… Huge improvements for syslog-ng in MacPorts ------------------------------------------- Recently I wrote about a campaign that we started to resolve issues on GitHub. Some of the fixes are coming from our enthusiastic community. Thanks to this, there is a new syslog-ng-devel port in MacPorts, where you can enable almost all syslog-ng features even for older MacOS versions and PowerPC hardware. Some of the freshly enabled modules include support for Kafka, GeoIP or OpenTelemetry. From this blog entry, you can learn how to install a legacy or an up-to-date syslog-ng version from MacPorts. https://www.syslog-ng.com/community/b/blog/posts/huge-improvements-for-sysl… WEBINARS * New webcast: “Log management superpowers: a PAM Essentials case study”. You can watch it at https://www.syslog-ng.com/webcast-ondemand/log-management-superpowers-a-pam… * You can learn about upcoming webinars and browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

can kv-parser options be from macros
by Evan Rempel 02 Nov '24

02 Nov '24

Does anyone know if I can use the enriching data to set macros that can then be used as kv-parser options. These values might change on a message by message basis. For example: parser p_kv { kv-parser ( prefix("$kv_prefix") value-separator("$kv_vseparator") pair-separator("$kv_pseparator") ); }; -- Evan

2 2

certificate errors result in excessive logging.
by Evan Rempel 21 Oct '24

21 Oct '24

I am using tls configuration with a locally signed certificate. This mans that I have to configure a custom root CA on to all client systems for them to be able to establish the tls connection to my syslog server. When the clients are unable to verify the server certificate, the server logs three messages for every connection attempt syslog-ng[452597]: SSL error while reading stream; tls_error='error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca', location='/etc/syslog-ng/syslog-ng.server.conf:71:17' syslog-ng[452597]: Error reading RFC6587 style framed data; fd='21', error='Connection reset by peer (104)' syslog-ng[452597]: Syslog connection closed; fd='21', client='AF_INET(1.2.3.4:1234)', local='AF_INET(1.2.3.4:1234)' If there are 100's of clients, and they try to reconnect at a fast rate (every 5 seconds) this can result in a large volume of messages. Is there any way to configure the logging rate of these types of errors or get rid of it altogether. Anyone have any comments on this? -- Evan

2 3

Re: [syslog-ng] syslog-ng Digest, Vol 234, Issue 4
by brian hoffman 15 Oct '24

15 Oct '24

Yes, you were spot on. I realized not long after I sent this that the .0 was missing on the end.Thanks!-Brian Date: Tue, 15 Oct 2024 00:32:37 +0000 From: Evan Rempel <erempel(a)uvic.ca> To: "syslog-ng(a)lists.balabit.hu" <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] netmask filter issue Message-ID: <YT4P288MB03355F0F965988C4C22978C0B9452(a)YT4P288MB0335.CANP288.PROD.OUTLOOK.COM> Content-Type: text/plain; charset="iso-8859-1" First off, to write a CIDR address correctly you need to use something of the form w.x.y.z/M Your specification is missing the ".z" prtion of the ip address. I think there is also some confusion regarding how netmasks work. The ip address 192.168.128.0 (I assume you want a .0 at the end) written in binary is 11000000.10101000.10000000.00000000 The /17 subnet mask written in binary is 11111111.11111111.10000000.00000000 If we perform a binary AND of these two addresses we get 11000000.10101000.10000000.000000 written in decimal is 192.168.128.0 This means that the FIRST address of this range is 192.168.128.0 If we perform the binary NOT of the subnet mask we get 00000000.00000000.01111111.11111111 When we take the binary OR of this with the IP address (w.x.y.z) we get 11000000.10101000.11111111.1111111 in decimal is 192.168.255.255 which is the last address if the range. The result is that 192.168.1.0/17 means the range 192.168.128.0 -- 192.168.255.255 I hope that helps. -- Evan

1 0

netmask filter issue
by brian hoffman 15 Oct '24

15 Oct '24

I set up a filter to send certain messages matching an IP range to a destination (a simplified version of my actual config).filter f_siem { netmask(192.168.128/17) or}; For some reason this is picking up the address 192.168.63.41, even though it's not in the range. 192.168.63.41 (10101100.00010111.0) does not match the network portion of 192.168.128.0/17 (10101100.00010111.1) but for some reason it's still getting picked up. I know for sure that it is this particular line because when I comment it out the messages are no longer forwarded for that device. Is there something different about the way this netmask directive works in syslog-ng that would cause it to get matched by this filter?-Brian

2 1