- syslog-ng - lists.balabit.hu

Re: [syslog-ng] Abort message processing - multiple includes
by Matthias Gruber 19 Nov '19

19 Nov '19

Hi! Good Lord, I think I just found the answer to my question in the Documentation :-/ :-) "This default behavior can be changed using the flags() parameter. Flags apply to individual log paths, they are not global options. " Therefore I think the final breaks out of the log-processing of the actual log-directive, but others "matching" will processed Argh... think I have to cat some of my config-files together... Sorry for "spamming" :-) cheers Matthias P.S. If I am wrong, please correct me (hope dies last ;->) ------------------------------------------------------------------------------------ METZLER Informationstechnologie Matthias Gruber IT-Infrastruktur & -Betrieb B. Metzler seel. Sohn & Co. Kommanditgesellschaft auf Aktien Untermainanlage 1 60329 Frankfurt am Main Telefon (0 69) 21 04 - 43 30 Telefax (0 69) 21 04 - 40 40 MGruber(a)metzler.com www.metzler.com Von: Matthias Gruber/METZLER An: "Syslog-ng users' and developers' mailing list" <syslog-ng(a)lists.balabit.hu> Datum: 19.11.2019 08:54 Betreff: Abort message processing - multiple includes Hi! Perhaps I misunderstood it, or got a bit crossed up, because I dig into syslog-ng-Configs since weeks, but I am unsure about the "flags(final)" statement I have several includes with several log-statements and several sources e.g. (no runnable syntax, only for demonstration-purpose) 0001-A.conf log { source(s_tcp_A); filter(f_filter_A); destination(d_somewhere_A); flags(final); }; 0002-B.conf log { source(s_tcp_A); destination(d_somewhere_BBB); flags(final); }; As I understand it, first of all, its included in "alphabetical order", so in my case first 0001-A.conf, then 0002-B.conf and so on.... But now my "Problem" and question (Yes I would normally do a "if"-Construct to solve that, problem above, but I used that example to understand the flags(final),) If I have a message received by s_tcp_A, filter f_filter_A hits, the rule sends it to d_somewhere_A and do a "Escape" cause of flags(final), does the message still got processes by the include of 0002-B.conf?? Does the flags(final) is only relevant for one "log-rule", or does it ends the whole processing? To be honest, I habe several includes, and a "last" one, which is imported last, with a "hit"-all, in my childish thought, if the flags(final) stopps processing of the message compleetly this hit-all only gets "the junk" where nothing else hists. Why do I do that, and not write it in one file with a large if-elif-else-construct, I do that for the readability since this file would be extremly large. I am thankfull for any help cheers Matthias ------------------------------------------------------------------------------------ METZLER Informationstechnologie Matthias Gruber IT-Infrastruktur & -Betrieb B. Metzler seel. Sohn & Co. Kommanditgesellschaft auf Aktien Untermainanlage 1 60329 Frankfurt am Main Telefon (0 69) 21 04 - 43 30 Telefax (0 69) 21 04 - 40 40 MGruber(a)metzler.com www.metzler.com Persönlich haftende Gesellschafter: Harald Illy, Michael Klaus, Friedrich von Metzler, Emmerich Müller, Gerhard Wiesheu Vorsitzender des Aufsichtsrats: Dr. Christoph Schücking Sitz der Gesellschaft: Frankfurt am Main, Handelsregister-Nr. HRB 27 515 Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfänger sein, so bitten wir Sie höflich, dies unverzüglich dem Absender mitzuteilen und die Nachricht zu löschen. Es ist unzulässig, die Nachricht unbefugt weiterzuleiten oder zu kopieren. Da wir nicht die Echtheit oder Vollständigkeit der in dieser Nachricht enthaltenen Informationen garantieren oder zusichern können, sind die vorstehenden Ausführungen rechtlich nicht bindend. Eine Haftung hierfür wird ausgeschlossen. This message is confidential. If you are not the intended recipient, we kindly ask you to inform the sender and delete the information. Any unauthorised dissemination or copying hereof is prohibited. As we cannot guarantee or assure the genuineness or completeness of the information contained in this message, the statements set forth above are not legally binding. Accordingly we cannot accept any liability for their contents.

1 0

Abort message processing - multiple includes
by Matthias Gruber 19 Nov '19

19 Nov '19

Hi! Perhaps I misunderstood it, or got a bit crossed up, because I dig into syslog-ng-Configs since weeks, but I am unsure about the "flags(final)" statement I have several includes with several log-statements and several sources e.g. (no runnable syntax, only for demonstration-purpose) 0001-A.conf log { source(s_tcp_A); filter(f_filter_A); destination(d_somewhere_A); flags(final); }; 0002-B.conf log { source(s_tcp_A); destination(d_somewhere_BBB); flags(final); }; As I understand it, first of all, its included in "alphabetical order", so in my case first 0001-A.conf, then 0002-B.conf and so on.... But now my "Problem" and question (Yes I would normally do a "if"-Construct to solve that, problem above, but I used that example to understand the flags(final),) If I have a message received by s_tcp_A, filter f_filter_A hits, the rule sends it to d_somewhere_A and do a "Escape" cause of flags(final), does the message still got processes by the include of 0002-B.conf?? Does the flags(final) is only relevant for one "log-rule", or does it ends the whole processing? To be honest, I habe several includes, and a "last" one, which is imported last, with a "hit"-all, in my childish thought, if the flags(final) stopps processing of the message compleetly this hit-all only gets "the junk" where nothing else hists. Why do I do that, and not write it in one file with a large if-elif-else-construct, I do that for the readability since this file would be extremly large. I am thankfull for any help cheers Matthias ------------------------------------------------------------------------------------ METZLER Informationstechnologie Matthias Gruber IT-Infrastruktur & -Betrieb B. Metzler seel. Sohn & Co. Kommanditgesellschaft auf Aktien Untermainanlage 1 60329 Frankfurt am Main Telefon (0 69) 21 04 - 43 30 Telefax (0 69) 21 04 - 40 40 MGruber(a)metzler.com www.metzler.com Persönlich haftende Gesellschafter: Harald Illy, Michael Klaus, Friedrich von Metzler, Emmerich Müller, Gerhard Wiesheu Vorsitzender des Aufsichtsrats: Dr. Christoph Schücking Sitz der Gesellschaft: Frankfurt am Main, Handelsregister-Nr. HRB 27 515 Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfänger sein, so bitten wir Sie höflich, dies unverzüglich dem Absender mitzuteilen und die Nachricht zu löschen. Es ist unzulässig, die Nachricht unbefugt weiterzuleiten oder zu kopieren. Da wir nicht die Echtheit oder Vollständigkeit der in dieser Nachricht enthaltenen Informationen garantieren oder zusichern können, sind die vorstehenden Ausführungen rechtlich nicht bindend. Eine Haftung hierfür wird ausgeschlossen. This message is confidential. If you are not the intended recipient, we kindly ask you to inform the sender and delete the information. Any unauthorised dissemination or copying hereof is prohibited. As we cannot guarantee or assure the genuineness or completeness of the information contained in this message, the statements set forth above are not legally binding. Accordingly we cannot accept any liability for their contents.

1 0

log server duplication
by freebsd＠tango.lu 15 Nov '19

15 Nov '19

Hello, I have a syslogNG based siem setup with customized rules like: options { use_dns(no); use_fqdn(no); check_hostname(no); owner(root); group(root); perm(0640); dir_owner(root); dir_group(root); dir_perm(0750); create_dirs(yes); normalize_hostnames(yes); keep_hostname(yes); # disable stats stats_freq(0); }; destination d_net_auth { file("/var/log/corporate/$HOST_FROM/auth.log"); }; ... These settings will not do dns resolution will result that when hosts sending their logs into this SIEM directories will be created by their IP addresses where the logs go. I would like to replicate this server on a second location without using brute methods like rsyncing the whole directory structure daily. I have configured syslogng to keep forwarding the logs to a remote destination which works fine however I can't select the messages based on the same criteria on the new log server because if I use the same config everything will originate from the IP for logserver 1. I need IP based directories on the second loghost as well, everything to be identical. I'm using syslogng 3.12. Is there a workaround for this? Thanks

3 4

Re: [syslog-ng] periodic disconnect of TCP sessions
by Wilson, Jonathan 15 Nov '19

15 Nov '19

Laci, thank you for your thoughts and questions. Some answers: My organization controls all the clients, but there are a _lot_ of them. Most are network devices or applications. Yes, the reason for multiple instances is HA. We have one in each of our data centers. A hardware load balancer provides the DNS entry and resolution (a "wide IP"), and it IS aware of the states of the servers - it checks both every few seconds to ensure that it can establish a TCP connection. If it cannot, the DNS name will not be switched to that server's IP. System load is not high. Both are VMs with multiple (virtual) CPUs, and syslog-ng rarely uses more than 10% of CPU. We get about 10k UDP messages per minute and about 1000 over TCP. None of the usual stress indicators - no UDP Rcvbuf errors, no bottlenecks that I can see. When initially set up, having two copies of every syslog was not a problem - it's just disk space. But when we starting indexing them in Splunk, it became a concern since Splunk is licensed by the volume of data indexed. Concurrent connections - less than 20 TCP connections at a time. I suppose what I'm looking for is a more granular load balancing, if that makes sense. With UDP, it's very granular, since each packet is sent statelessly to wherever the "wide IP" points at that second. But with TCP, long-lived connections are the rule - great for throughput, but they kind of defeat the purpose. If syslog-ng kept a count of messages received over each TCP connection and did an orderly shutdown after "n" had been received, the client would reconnect seamlessly and all would be well. Alternatively, shut down a TCP connection after it had been open for "n" seconds. Thanks, Jon Message: 1 Date: Fri, 15 Nov 2019 10:16:18 +0000 From: "Laszlo Szemere (lszemere)" <Laszlo.Szemere(a)oneidentity.com> To: "syslog-ng(a)lists.balabit.hu" <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] periodic disconnect of TCP sessions Message-ID: <BL0PR1901MB20170F90FB51A130781F3F429D700(a)BL0PR1901MB2017.namprd19.prod.outlook.com> Content-Type: text/plain; charset="Windows-1252" Hello Jon, There is currently no way to deliberately terminate an active TCP session. What I was thinking about, that there might be a better approach to the original problem, and terminating the TCP session is just a fragile workaround. I have a couple of questions about your setup, so I can get a better understanding of the problem: - Is all of the clients controlled by you? (Maybe they are syslog-ng instances in the first place?) - Is there a particular reason to have multiple syslog-ng instances? At a first glance (with the parallel sending) it looked like a HA(ish) setup, but the later round-robin solution ruled it out. Also it looks like the DNS server is not aware of the states of the servers. - What is the current load on the system? Have you experienced any throttling or bottle neck. (Why do I ask? If the reason behind using multiple syslog-ng instances, is to load balance the traffic for the Splunk servers, than it can be achieved in a different way. - How many concurrent connections do you expect at the peak load? Best regards, Laci ________________________________________ From: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu> on behalf of Wilson, Jonathan <jonathan.wilson(a)vumc.org> Sent: Wednesday, November 13, 2019 23:03 To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] periodic disconnect of TCP sessions CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Hello all, I am using a pair of syslog-ng OSE 3.22.1 servers that write logfiles which are then scanned by a Splunk Universal Forwarder. They receive messages over TCP, TCP with TLS, and UDP. We have always had devices and systems that send us syslog messages simply send to both syslog-ng servers; however, this resulted in double indexing of the log data in Splunk. To deal with this we set up a DNS name that round-robins across the two syslog-ng servers’ IPs every 30 seconds. The devices and systems that send to us now send to that DNS name. That neatly prevents the double indexing. If the messages are coming in over stateless UDP, the messages are load balanced in that they all go to one server for 30 seconds, then the other. However, TCP sessions are much longer lived, and some senders send many messages every second – they will latch onto one of our syslog servers and stay connected to it all day. What I am looking for is a way to limit the lifetime of a TCP connection into syslog-ng, either by time or by number of messages received; after the connection is dropped, the sender will reconnect to whichever server is indicated by the round-robin DNS name, and over time about half of the messages will go to each server. Is there already a way to do this? Failing that, can you suggest a place to start in patching the source? Thanks, Jon

1 0

periodic disconnect of TCP sessions
by Wilson, Jonathan 15 Nov '19

15 Nov '19

Hello all, I am using a pair of syslog-ng OSE 3.22.1 servers that write logfiles which are then scanned by a Splunk Universal Forwarder. They receive messages over TCP, TCP with TLS, and UDP. We have always had devices and systems that send us syslog messages simply send to both syslog-ng servers; however, this resulted in double indexing of the log data in Splunk. To deal with this we set up a DNS name that round-robins across the two syslog-ng servers' IPs every 30 seconds. The devices and systems that send to us now send to that DNS name. That neatly prevents the double indexing. If the messages are coming in over stateless UDP, the messages are load balanced in that they all go to one server for 30 seconds, then the other. However, TCP sessions are much longer lived, and some senders send many messages every second - they will latch onto one of our syslog servers and stay connected to it all day. What I am looking for is a way to limit the lifetime of a TCP connection into syslog-ng, either by time or by number of messages received; after the connection is dropped, the sender will reconnect to whichever server is indicated by the round-robin DNS name, and over time about half of the messages will go to each server. Is there already a way to do this? Failing that, can you suggest a place to start in patching the source? Thanks, Jon

2 1

Insider 2019-11: logging to Elasticsearch; PE 6 to 7 upgrade; Elastic 7; in-list(); off-line deb; Splunk conf;
by Peter Czanik (pczanik) 14 Nov '19

14 Nov '19

Dear syslog-ng users, This is the 76th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Logging to Elasticsearch made simple with syslog-ng --------------------------------------------------- Elasticsearch is gaining momentum as the ultimate destination for log messages. You can store arbitrary name-value pairs coming from structured logging or message parsing and you can use Kibana as a search and visualization interface. You can use syslog-ng as an all in one solution (system and application logs, message parsing, filtering and queuing) to collect and forward log messages to Elasticsearch: https://www.syslog-ng.com/community/b/blog/posts/logging-to-elasticsearch-m… Upgrading syslog-ng PE from version 6 to 7 ------------------------------------------ Learn the major steps necessary to upgrade your system from syslog-ng Premium Edition version 6 to 7. As you will see, it is no more difficult than any other major software version upgrade, and after the upgrade you can start using all the new and useful features that are available in version 7. https://www.syslog-ng.com/community/b/blog/posts/upgrading-syslog-ng-pe-fro… syslog-ng and Elasticsearch 7: getting started on RHEL/CentOS ------------------------------------------------------------- Version 7 of the Elastic Stack, packed with new features and improved performance, has now been available for some time. Elasticsearch is not the only one to have come up with a major new version recently: starting with version 3.21, syslog-ng features a new Elasticsearch destination driver (based on the http() destination) that does not require Java. In most cases, it is more resource friendly than the Java-based driver and it is definitely easier to configure. It also has the benefit that unlike the Java-based driver, it can be included in Linux distributions. This is a quick how-to guide to get you started with syslog-ng and Elasticsearch 7 on RHEL/CentOS 7. https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-and-elasticsearc… Handling lists in syslog-ng: the in-list() filter ------------------------------------------------- Recently, a number of quite complex configurations came up while syslog-ng users were asking for advice. Some of these configurations were even pushing the limits of syslog-ng (regarding the maximum number of configuration objects). As it turned out, these configurations could be significantly simplified using the in-list() filter, one of syslog-ng’s lesser known features. https://www.syslog-ng.com/community/b/blog/posts/handling-lists-in-syslog-n… Offline syslog-ng DEB package installer --------------------------------------- “How can I install the unofficial syslog-ng packages on a machine without Internet access?” This question has been raised several times recently. As it entails more than simply downloading the repository containing the packages, syslog-ng lead developer Laszlo Budai created a script that solves the problem for Debian and Ubuntu users. The script downloads syslog-ng along with its dependencies using a container, and produces an archive containing all DEB packages necessary to install syslog-ng as well as a simple script responsible for the installation. Using a container in this case might seem like over complication, but it is still the easiest way to ensure that all dependencies are included in the archive. https://www.syslog-ng.com/community/b/blog/posts/offline-syslog-ng-deb-pack… syslog-ng at Splunk .conf 2019 ------------------------------ The syslog-ng team participated at Splunk .conf 2019 and came home with positive experiences. They met enthusiastic users, who use syslog-ng in front of Splunk as log management layer. Many of them already using the HTTP event collector and use syslog-ng to load-balance log messages among collectors. They also demonstrated how syslog-ng can collect over one million log messages a second over UDP. https://twitter.com/OneIdentity/status/1187852500222693377 WEBINARS * You can achieve 1m+ UDP messages per second ingestion challenge: https://www.syslog-ng.com/webcast-ondemand/you-can-achieve-1m-udp-messages-… * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

3 3

syslog-ng in EPEL 8
by Peter Czanik (pczanik) 13 Nov '19

13 Nov '19

Hi, Version 3.23.1 of syslog-ng is now available in EPEL 8. It has most, but not all of the syslog-ng features available. Missing features: * kafka destination, C implementation, as librdkafka is too old * Java destination drivers (kafka, elastic, hdfs), as sources necessary to compile them are not in the distro. Note, that Elastic is supported through elastic-http() * tcp wrapper, as support was removed from RHEL * riemann does not have sub-second precision, as client in EPEL is too old As missing dependencies arrived to EPEL 8, right now the official EPEL 8 syslog-ng package has more features enabled, than my own experimental EPEL 8 package 🙂 Once 3.25.1 is released, I hope to support RHEL/CentOS 8 from my regular syslog-ng repositories. The difference is not as big as it was before missing dependencies became available, but I still need to figure out some RPM spec magic 🙂 EPEL 8 stays at 3.23 (no version update is allowed there). There are plans to allow new versions into EPEL with each new minor RHEL release, but seeing is believing. It was promised for too many years. Bye, Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

2 7

parse rancher logs from syslog-ng files
by Tech Gurus 13 Nov '19

13 Nov '19

Hi , I have setup where rancher 2.3 send logs ogs to syslog-ng server , I am getting several GB per hour and write them to text file. The log format are dynamic , general form are [timestamp SourceContainer-host-IP Source-Program list-of-nested-json-key/value-pairs] .. I got the elasticsearch forwarding wokring with config below, however all json kv gets written to message field.. I need to be able to : * Parse json kv into diffeent fields to elastic and flatten the json fields. * Use one field to create the index on destination elastic like "program" or custom added field on source side like "k8s-cluster-name" Thanks for help John destination d_elasticsearch_http { elasticsearch-http( url("https://elastricsearch:9200/_bulk") template("$(format-json --scope rfc5424 --scope dot-nv-pairs --rekey .* --shift 1 --scope nv-pairs --exclude DATE --key ISODATE @timestamp=${ISODATE})") index("syslog-ng-test") #time-zone("UTC") type("") workers(4) batch-lines(100) #batch-timeout(10000) #timeout(10) tls( #ca-dir("/etc/syslog-ng/conf.d") ca-file("/etc/syslog-ng/conf.d/root-ca.pem") cert-file("/etc/syslog-ng/conf.d/admin.pem") key-file("/etc/syslog-ng/conf.d/admin.key") peer-verify(no) ) ); };

2 1

Building syslog with cmake as external project
by Alexandre Santos 12 Nov '19

12 Nov '19

Hi All, I am trying to build syslog-ng with cmake to add it to a custom Linux distribution. The problem is that when the syslog-ng is compiled it gets the RPATH of the location where it has been generated: *root@5526d87140af:/home/src/**chassisx/build# ldd syslog-ng-install/sbin/syslog-* *ng linux-vdso.so.1 (0x00007fff151ed000) libsyslog-ng.so.3.24.1 => /home/src/target/build/**syslog-ng-install/lib/* *libsyslog-ng.so.3.24.1 (0x00007f81baf89000) libeventlog.so => /home/src/target/build/**syslog-ng-install/lib/* *libeventlog.so (0x00007f81bad84000) libgmodule-2.0.so.0 => /usr/lib/x86_64-linux-gnu/* *libgmodule-2.0.so.0 (0x00007f81bab80000) libgthread-2.0.so.0 => /usr/lib/x86_64-linux-gnu/* *libgthread-2.0.so.0 (0x00007f81ba97e000) libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-* *2.0.so.0 (0x00007f81ba66a000) libivykis.so.0 => /home/src/target/build/**syslog-ng-install/lib/* *libivykis.so.0 (0x00007f81ba45b000) libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.* *so.3 (0x00007f81ba1e8000) libresolv.so.2 => /lib/x86_64-linux-gnu/* *libresolv.so.2 (0x00007f81b9fd1000) libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/* *libssl.so.1.1 (0x00007f81b9d65000) libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/* *libcrypto.so.1.1 (0x00007f81b98cc000) libsecret-storage.so => /home/src/target/build/**syslog-ng-install/lib/* *libsecret-storage.so (0x00007f81b96c8000) libpthread.so.0 => /lib/x86_64-linux-gnu/* *libpthread.so.0 (0x00007f81b94ab000) libc.so.6 => /lib/x86_64-linux-gnu/libc.so.* *6 (0x00007f81b910c000) /lib64/ld-linux-x86-64.so.2 (0x00007f81bb48e000) libdl.so.2 => /lib/x86_64-linux-gnu/libdl.**so.2 (0x00007f81b8f08000)* I am using the cmake file in attachment. Can you help me? Thanks in advance, Alex

2 4

Insert SequenceNumber into FileName
by William Luiz Ribeiro Vasconcelos Da Silva 12 Nov '19

12 Nov '19

Good Morning Is this possible put some kind of sequence Number or some kind ID number at the fileName, when the syslog-ng is writing the file???? If it's possible, how to do this? Tks, Atenciosamente, WILLIAM LUIZ R V SILVA Mediation Ericsson, Brazil Mobile +55 11 97979-9886 wsilva_ericsson(a)timbrasil.com.br<mailto:wsilva_ericsson@timbrasil.com.br> www.ericsson.com<http://www.ericsson.com/> [Descri??o: Descri??o: Ericsson]<http://www.ericsson.com/> Esta mensagem, incluindo seus anexos, pode conter informa??es privilegiadas e/ou de car?ter confidencial, n?o podendo ser retransmitida sem autoriza??o do remetente. Se voc? n?o ? o destinat?rio ou pessoa autorizada para receb?-la, informamos que o seu uso, divulga??o, c?pia ou arquivamento s?o proibidos. Portanto, se voc? recebeu esta mensagem por engano, por favor nos informe respondendo imediatamente a este e-mail e delete o seu conte?do. This message, including its attachments, may contain privileged or confidential information, and it must not be fowarded without the express authorization of the sender. If you are not the intended recipient, we hereby inform you that the use, disclosure, copy or filing are forbidden. So, if you received this message as a mistake, please inform us by answering this e-mail and deleting its contents Questo messaggio, inclusi gli allegati, potrebbe contenere informazioni privilegiate e/o riservate, e non deve essere ritrasmesse senza l'autorizzazione del mittente. Se non siete il destinatario o la persona autorizzata a riceverlo, informiamo che il suo utilizzo, diffusione, copia o archiviazione sono proibite. Quindi, se avete ricevuto questo messaggio per errore, per cortesia ci informi rispondendo immediatamente a questa email e cancelli il suo contenuto

2 1