- syslog-ng - lists.balabit.hu

Requesting help with Grouping-by function
by Maciek Solnicki 04 Nov '20

04 Nov '20

Hi All, I would like to ask for help with usage of grouping-by function. Goal is to correlate logs from auditd. Logs produced by auditd are multiline, but events share some number in key with name *msg. *This in theory should allow aggregation of logs with the same *msg *key, but I cannot get it right. Here is my config: *log {* * source {* * file('/var/log/audit/audit.log' flags(no-parse));* * };* * parser {* * linux-audit-parser(prefix(".auditd."));* * };* * parser {* * grouping-by(* * key("${.auditd.msg}")* * timeout(10)* * aggregate(value("MESSAGE" "TEST"))* * );* * };* * destination {* * file('/tmp/test.json' template("$(format-json .auditd.*)\n"));* * };* *};* After implementing above config, output logs look like this, prefix is added as expected and format is set o JSON, but no aggregation happens: *{"_auditd":{"type":"EXECVE","msg":"audit(1604391099.830:195558):","argc":"2","a1":"-F","a0":"/usr/sbin/syslog-ng"}}* *{"_auditd":{"type":"CWD","msg":"audit(1604391099.830:195558):","cwd":"/"}}* *{"_auditd":{"type":"PATH","rdev":"00:00","ouid":"0","ogid":"0","nametype":"NORMAL","name":"/usr/sbin/syslog-ng","msg":"audit(1604391099.830:195558):","mode":"0100755","item":"0","inode":"1078935","dev":"08:01","cap_fver":"0","cap_fp":"0000000000000000","cap_fi":"0000000000000000","cap_fe":"0"}}* *{"_auditd":{"type":"PATH","rdev":"00:00","ouid":"0","ogid":"0","nametype":"NORMAL","name":"/lib64/ld-linux-x86-64.so.2","msg":"audit(1604391099.830:195558):","mode":"0100755","item":"1","inode":"1048592","dev":"08:01","cap_fver":"0","cap_fp":"0000000000000000","cap_fi":"0000000000000000","cap_fe":"0"}}* *{"_auditd":{"type":"PROCTITLE","proctitle":"/usr/sbin/syslog-ng\t-F","msg":"audit(1604391099.830:195558):"}}* Syslog-ng version I'm using: *syslog-ng 3 (3.29.1)* *Config version: 3.29* *Installer-Version: 3.29.1* *Revision: 3.29.1-1* *Compile-Date: Aug 28 2020 12:13:25* *Module-Directory: /usr/lib/syslog-ng/3.29* *Module-Path: /usr/lib/syslog-ng/3.29* *Include-Path: /usr/share/syslog-ng/include* *Available-Modules: pseudofile,cef,confgen,stardate,tfgetent,kvformat,afmongodb,afamqp,afsocket,system-source,csvparser,geoip2-plugin,syslogformat,graphite,affile,xml,dbparser,afprog,add-contextual-data,mod-python,sdjournal,map-value-pairs,timestamp,pacctformat,disk-buffer,afstomp,afsmtp,basicfuncs,afuser,tags-parser,linux-kmsg-format,hook-commands,examples,appmodel,afsql,cryptofuncs,json-plugin,riemann,redis* *Enable-Debug: off* *Enable-GProf: off* *Enable-Memtrace: off* *Enable-IPv6: on* *Enable-Spoof-Source: on* *Enable-TCP-Wrapper: on* *Enable-Linux-Caps: on* *Enable-Systemd: on* Can anyone please tell what am I doing wrong ? Kind regards Maciej

4 8

Performance issue with python code in template
by Diego Billi 04 Nov '20

04 Nov '20

Hi, I have a performance problem. This is a skeleton of my syslog-ng configuration. >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> python { import ...mylibs... def t_my_python_function(msg): ... output_data = .... process msg object ... ... return output_data } source s_mysource { udp( .... ); } destination d_mydestination { file( "/tmp/mylogs.log" template("$(python t_my_python_function)") ); }; log { source(s_mysource); destination(d_mydestination); flags(flow-control); }; <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< I receive syslog messages via UDP with a very very high rate. Incoming messages are processed with a template written in python. Syslog-ng have performance issues with this flow due to the python code. I'm wondering how to speed up this solution. Note that the python function can be parallelized (no shared state/data between messages) I'm trying this solution but i don't know if it changes that much. ------------------------------------------------------------------------ python { import ...mylibs... def p_my_python_function(msg): ... ... output_data = .... process msg object ... ... ... msg['MY_OUTPUT_DATA'] = output_data } source s_mysource { ... } destination d_mydestination { file( "/tmp/mylogs.log" template("${MY_OUTPUT_DATA}") ); }; log { source(s_mysource); parser(p_my_python_function); <---- moved here (outside destination) destination(d_mydestination); flags(flow-control); }; ------------------------------------------------------------------------ Moving the "processing" outside the "destination" is really useful? I'm trying to understand if i can use threading and multi-core supporto of syslog-ng. Thank you for your time. Diego.

3 2

Recommended db for syslog?
by victorsugo 26 Oct '20

26 Oct '20

Hi, I'd like to store my logs in a database, instead of text files. Many years ago I thought a mysql or postgresql can be excellent for that purpose, but today... I'm a little bit confused. Should I use an RDBMS? Or should I turn to noSQL databases? (mongoDB?) The only thing I'm sure, I don't want to use any other log manager(??) software (like Graylog, elasticsearch etc.) I'd like to use only a syslog-ng server with a database. Could you help me? Victor :)

1 0

Insider 2020-10: Cisco; Signal Messenger; PCRE dupnames;
by Peter Czanik (pczanik) 08 Oct '20

08 Oct '20

Dear syslog-ng users, This is the 85th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Parsing Cisco logs in syslog-ng -------------------------------- Log messages generated by Cisco devices look like syslog messages at first glance, but on a closer inspection you will see that there are many smaller differences. By default, syslog-ng treats all incoming messages as syslog messages, however, Cisco logs do not conform. Log messages collected over the network from Cisco devices and saved to a file look broken. There are many Cisco log variants but luckily a good part of them are covered by the cisco-parser() of syslog-ng.From this blog you can learn how the Cisco parser in syslog-ng works and how you can check if it really works with your Cisco log messages. https://www.syslog-ng.com/community/b/blog/posts/parsing-cisco-logs-in-sysl… Sending alerts to Signal Messenger from syslog-ng ------------------------------------------------- Signal Messenger is becoming the instant messaging platform of choice for privacy-minded individuals, including many sysadmins. No wonder that some of them would like to see alerts from syslog-ng in this IM platform. Developing a new destination for syslog-ng from scratch in the C programming language is a considerable effort. As a result, this first implementation is utilizing an already existing command line application. Below, you can learn about an initial implementation, and why it is not part of syslog-ng. https://www.syslog-ng.com/community/b/blog/posts/sending-alerts-to-signal-m… Enabling PCRE dupnames in syslog-ng ----------------------------------- One of the major syslog-ng features is that it can parse log messages and create name-value pairs from them. Until now the PCRE parser could not handle duplicate names for named subpatterns. Version 3.29 of syslog-ng resolves this issue by adding the “dupnames” flag. From this blog you can learn why the dupnames flag is important and how you can enable and test it. https://www.syslog-ng.com/community/b/blog/posts/enabling-pcre-dupnames-in-… WEBINARS * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

squid access.log to syslog-ng - how?
by victorsugo 06 Oct '20

06 Oct '20

Hi, I'd like to send all of my squid proxy's log to a syslog-ng server. But it doesn't work as I want, because when syslog-ng parses the lines from access.log, it interprets the first field (actually UTC timestamp) as a program. (I get the timestamp value in the PROGRAM field). But... if I put an rsyslog (installed locally on squid's container) between the squid and syslog-ng servers, I can get correct results. Could you help me, how can I omit the rsyslog from the logging and get a usable log? Is there a standard way to do it? Or do I need to create an own template in squid.conf for this log? regards, Victor --------------------- What's the vector Victor? :) ---------------------------------------------------

2 1

rewrite error
by operacoes - mls wireless 05 Oct '20

05 Oct '20

Hello there, I am runing syslog-ng 3.2.5 on a centos 6.10 server, the log server. Here we have 30 clients sending log via rsyslog to the log server. At the log server each one of them have its own log according to the host name. This operation works just fine. Now my manager ask me to remove some parts of the log, so the final log size will be smaller. At the log server I tried use rewrite and unset to suppress some parts of the log but it does not work, with the error : Plugin module not found in 'module-path'; module-path='/lib64/syslog-ng', module='rewrite' This is the config on log server for each one of the clients. filter f_client_serv01 { host( "client_serv01" ); }; destination d_client_serv01 { file("/var/log/client_serv01.log" perm(0644) owner(root) group(root) ); }; log { source(s_net); filter(f_client_serv01); destination(d_client_serv01); }; This is the log Oct 2 19:07:04 client_serv01 kernel: IN=rdc1 OUT= MAC=00:50:da:6f:0a:54:ec:08:6b:7f:d5:cb:08:00 SRC=10.1.16.35 DST=xxx.xxx.xxx.xxx LEN=60 TOS=0x00 PREC=0x00 TTL=63 ID=27903 DF PROTO=TCP SPT=40917 DPT=443 WINDOW=65535 RES=0x00 SYN URGP=0 rewrite r_rewrite_unset{ unset(value("WINDOW")); }; rewrite r_rewrite_subst{ subst("WINDOW", " ", value("MESSAGE"), flags("global")); }; Does anyone have a tip to try to solve this? Thanks for help. Have a nice day! -- Para contato ou solicitações utilize nosso canal de atendimento via Suporte Técnico através de nossa central 24H via tel.:(21) 2538-8400 ou através do email: suporte(a)mls.com.br Atenciosamente, *João Tiago Braz*| Aux. Sistemas Operação 21 2538-8400 | 21 3923-5000 operacoes(a)mls.com.br

2 2

Rewrite JSON field without re-formating via custom template
by Nirgil 29 Sep '20

29 Sep '20

Hello all, sorry to bother you, but I'm bit lost ;] Trying to accomplish the following: Have a JSON inside syslog message and I need to rewrite, or better to remove, one selected particular field, it contains lot of data and I just want to drop this one field before forwarding to different destination. I have created a parser for JSON: parser p_json { channel { parser { json-parser (prefix(".json.")); }; }; }; Now I need remove or change a JSON field, lets say payload. So I created the following rewrite rule and tried multiple options to remove or change the content rewrite { unset(value("${.json.payload}")); unset(value(".json.payload")); groupunset(values("${.json.payload}")); set("TEST", value(".json.payload")); }; This is working on default fields ${HOST}, etc, but not for JSON fields. Message can be reformatted via template ie: template json_template { template("${DATE} ${HOST} $PROGRAM: {\"payload\"\:\""${.json.payload}"\"}") }; In that case, json.payload can be affected by appropriate rewrite rule. But I'd like to affect the json.payload field inside the original ${MSG} without re-formatting whole message via custom template, because the original message has lot of optional fields, other inner JSON's and it is not so simple to made an universal template for this. template json_template { template("${DATE} ${HOST} $PROGRAM: ${MSG}") }; Is there a way how to remove JSON field from original message (${MSG}) without reformatting the whole message via custom template ? Thanks for any thoughts ! Sample of Message: Sep 23 15:22:48 hostname myapplication: {"payload":"verylongpayload","field1":"value",.."lot of other fields, included inner JSON"...}

3 4

Syslog-NG within Kubernetes
by Allen Olivas 22 Sep '20

22 Sep '20

Hello all, I've been working on a containerized and orchestrated environment for a little while now (learning how it all works and slowly implementing it). One step I need to add is Syslog-NG. I"ve copied down the Syslog-NG Docker files and attempted adding these to a deployment in Kubernetes. Syslog-NG appears to be running successfully in Docker but in Kubernetes the logs I get say: syslog-ng: Error setting capabilities, capability management disabled; error='Operation not permitted' I'm not entirely sure where to turn too from this point. I've looked online for any ideas or resolution to this issue but so far non seem to work for my use case. I figured maybe someone on the mailing list might have an Idea? Specifically I'm trying to find out what that error means, and how best to orchestrate a containerized Syslog-NG. Thanks!

2 1

Can I manually setup a Syslog server?
by Turritopsis Dohrnii Teo En Ming 20 Sep '20

20 Sep '20

Subject: Can I manually setup a Syslog server? Good day from Singapore, I have just implemented and setup the following for an investment company (company name is confidential) in Singapore on 12 Sep 2020 Saturday Singapore Time. (1) Installed and configured Kiwi Free Syslog Server 9.7.0 (by Solarwinds) on the Active Directory Domain Controller (Windows Server 2016 Standard). (2) Configured Cisco ASA 5506-X Firewall's Logging to send and transmit syslog messages to Kiwi Free Syslog Server 9.7.0 on the Windows Server. (3) Added a firewall rule to open UDP port 514 on the Symantec Endpoint Protection Manager (SEPM) on Windows Server 2016. So the question I have is, can I manually setup and configure a Syslog Server with GUI using open source syslog daemon, open source tools and any Linux distro? Thank you very much. -- -----BEGIN EMAIL SIGNATURE----- The Gospel for all Targeted Individuals (TIs): [The New York Times] Microwave Weapons Are Prime Suspect in Ills of U.S. Embassy Workers Link: https://www.nytimes.com/2018/09/01/science/sonic-attack-cuba-microwave.html ******************************************************************************************** Singaporean Targeted Individual Mr. Turritopsis Dohrnii Teo En Ming's Academic Qualifications as at 14 Feb 2019 and refugee seeking attempts at the United Nations Refugee Agency Bangkok (21 Mar 2017), in Taiwan (5 Aug 2019) and Australia (25 Dec 2019 to 9 Jan 2020): [1] https://tdtemcerts.wordpress.com/ [2] https://tdtemcerts.blogspot.sg/ [3] https://www.scribd.com/user/270125049/Teo-En-Ming -----END EMAIL SIGNATURE-----

2 2

Insider 2020-09: Prometheus; proxy; ESK;
by Peter Czanik (pczanik) 09 Sep '20

09 Sep '20

Dear syslog-ng users, This is the 84th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Using a proxy with the http() destination ----------------------------------------- The http() destination is quickly becoming one of the most often used destinations within syslog-ng. You might already be using it even if you are not aware of it. Quite a few syslog-ng destination drivers are actually just configuration snippets in the syslog-ng configuration library (SCL), utilizing the http() destination in the background. Just think about elasticsearch-http(), different Logging as a Service (LasS) providers, or slack(). Starting with syslog-ng version 3.28.1 you can also reach these services when there is a proxy server between syslog-ng and your destination. https://www.syslog-ng.com/community/b/blog/posts/using-a-proxy-with-the-htt… Prometheus: syslog-ng exporter ------------------------------ Recently Prometheus became one of the most used open source monitoring solutions. Quite a few people asked if a syslog-ng exporter is available. It is not part of syslog-ng, but there are numerous implementations available on GitHub. Now that Prometheus is part of the openSUSE Leap 15.2 release, which is the Linux distribution running on my laptop, I gave it a try. From this blog, you can learn how to compile the syslog-ng exporter for Prometheus yourself and get it working with Prometheus. https://www.syslog-ng.com/community/b/blog/posts/prometheus-syslog-ng-expor… Jump-starting ESK: Elasticsearch, syslog-ng and Kibana ------------------------------------------------------ If you want to test drive syslog-ng or just want to learn something new, I recommend you checking out the BLACK ESK project. By running a single script, you can set up a containerized test environment, complete with Elasticsearch, Kibana and a syslog-ng server. All network connections among them are encrypted and the syslog-ng configuration showcases many interesting syslog-ng features, including PatternDB and JSON parsing, GeoIP, in-list filtering and the new Elasticsearch destination. Once it is installed, all you need are some logs directed at this server and a browser to reach Kibana. You can learn a lot from reading through the setup scripts and the different configuration files. https://www.syslog-ng.com/community/b/blog/posts/jump-starting-esk-elastics… NEW RELEASES * Version 3.29.1 released: https://github.com/syslog-ng/syslog-ng/releases/tag/syslog-ng-3.29.1 WEBINARS * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Multi-line messages across socket
by Alexandre Santos 07 Sep '20

07 Sep '20

Hi, I have some multi-line messages that are being broken, into single log messages, when they are sent over some remote host. In my configuration, I have one syslog-ng sending messages to other syslog-ng over unix-domain-socket. (check configurations in attachment). My goal is to have content of "/tmp/test1_udp_file.log" equal to the content of "/var/log/netconf-commands.log". Do know what am I missing? Any help is appreciated. Thanks in advance. Alex *</usr/sbin/syslog-ng -Fvde>* [2020-09-07T15:57:17.711583] Incoming log entry from journal; message='AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Test at 2020-09-07T15:57:17,448482284+00:00 \x0awith newline BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB' [2020-09-07T15:57:17.711760] json-parser(): no marker at the beginning of the message, skipping JSON parsing ; input='AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Test at 2020-09-07T15:57:17,448482284+00:00 \x0awith newline BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB', marker='@cim:' [2020-09-07T15:57:17.712016] Reliable disk-buffer state saved; filename='/tmp/syslog-ng-00002.rqf', qdisk_length='0' [2020-09-07T15:57:17.712046] Initializing destination file writer; template='/var/log/netconf-command.log', filename='/var/log/netconf-command.log' [2020-09-07T15:57:17.712150] Outgoing message; message='<158>1 2020-09-07T15:57:17.449+00:00 localhost root 25947 - - AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Test at 2020-09-07T15:57:17,448482284+00:00 \x0awith newline BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB\x0a' [2020-09-07T15:57:17.712299] Outgoing message; message='<158>1 2020-09-07T15:57:17.449+00:00 MYHOSTNAME root 25947 - - AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Test at 2020-09-07T15:57:17,448482284+00:00 \x0awith newline BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB\x0a' *</var/log/netconf-commands.log>* <158>1 2020-09-07T15:57:17.449+00:00 localhost root 25947 - - AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Test at 2020-09-07T15:57:17,448482284+00:00 with newline BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB *<ip vrf exec MGMT /usr/sbin/syslog-ng -Fvde --cfgfile=/etc/syslog-ng/mgmt-syslog-ng.conf --pidfile=/var/lib/syslog-ng/mgmt-syslog-ng.pid --persist-file=/var/lib/syslog-ng/mgmt-syslog-ng.persist --control=/var/lib/syslog-ng/mgmt-syslog-ng.ctl>* [2020-09-07T15:57:17.712514] Incoming log entry; line='<158>1 2020-09-07T15:57:17.449+00:00 MYHOSTNAME root 25947 - - AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Test at 2020-09-07T15:57:17,448482284+00:00 ' [2020-09-07T15:57:17.712762] Incoming log entry; line='with newline BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB' [2020-09-07T15:57:17.712997] Outgoing message; message='<158>1 2020-09-07T15:57:17.449+00:00 MYHOSTNAME root 25947 - [meta sequenceId="2"] AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Test at 2020-09-07T15:57:17,448482284+00:00 \x0a' [2020-09-07T15:57:17.713250] Outgoing message; message='<158>1 2020-09-07T15:57:17.449+00:00 MYHOSTNAME root 25947 - [meta sequenceId="2"] AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Test at 2020-09-07T15:57:17,448482284+00:00 \x0a' *</tmp/test1_udp_file.log>* <158>1 2020-09-07T15:57:17.449+00:00 MYHOSTNAME root 25947 - [meta sequenceId="2"] AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA Test at 2020-09-07T15:57:17,448482284+00:00

2 1

Re: [syslog-ng] message being consistently dropped
by Gabor Nagy (gnagy) 01 Sep '20

01 Sep '20

Hi, Your logs get dropped by the filter "facility(local1)". The filter does not match on the above log because it cannot be parsed by the syslog-protocol (RFC5424) due to the unescaped brackets in the SDATA. Due to the failed parsing, your log message has been altered to visualize the parsing errors location: <43>1 2020-09-01T18:28:24+02:00 localhost syslog-ng 2514 - - Error processing log message: <141>1 2020-08-04T08:10:58.769127-05:00 fqdn.example.com apmd 12374 01490113:5: [F5@12276 hostname="fqdn" errdefs_msgno="01490113:5:" partition_name="Common" session_id="1c95e1e7" Access_Profile="/Common/blah" Partition="Common" Session_Id="1c95e1e6" Session_Variable_Name="session.machine_info.last.net_adapter.list.[0>@<].mac_address" Session_Variable_Value="3C:D9:2B:33:9A:8E"] /Common/<blah>:Common:1c95e1e6: session.machine_info.last.net_adapter.list.[0].mac_address is 3C:D9:2B:33:9A:8E The question is, do you need to parse the message before storing it into a log file? Because we can disable parsing on the source side and the whole message is put into the $MESSAGE macro. I guess you need some basic parsing, as you filter the messages based on facility. Do you use this source in other log paths as well? I'm thinking in a workaround currently, as so far I didn't found a way to overcome this issue. I would like to emphasize that F5 log format violates the RFC5424 described protocol. This should be reported to them. Regards, Gabor ________________________________ From: Wilson, Jonathan <jonathan.wilson(a)vumc.org> Sent: Tuesday, September 1, 2020 14:39 To: Gabor Nagy (gnagy) <Gabor.Nagy(a)oneidentity.com> Subject: Re: [syslog-ng] message being consistently dropped CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Good morning Gabor, thank you for the info on my syslog-ng issue. Unfortunately the device in question is “canned” (an F5) and I am pretty sure I cannot change the format it logs with. Is there a simple way to pre-process them so that they don’t get dropped by syslog-ng? Thanks, Jon Jon Wilson | Principal System Engineer, IT Service Management | Information Technology | Vanderbilt University Medical Center jonathan.wilson(a)vumc.org<mailto:jonathan.wilson@vumc.org> | phone: 615-440-7895 | fax: 615-323-2181

1 0

Re: [syslog-ng] syslog-ng is ignoring a network source
by Bruns, Benjamin 01 Sep '20

01 Sep '20

Hello Laci, Thanks a lot! It turned out that my second firewall does not come through to netcat on port 514, even if I turned off the firewall completely. It's all running a Security Onion host and now I'm wondering if the IDS/IPS is fooling me and is blocking the traffic (but I don't notice any alerts related to the IP of the second firewall ...). However, any other ideas? Mit freundlichen Grüßen / Best regards Benjamin Bruns IT Security Manager CYPP GmbH Gotenstrasse 15 20097 Hamburg Germany Telefon: +49 40 237 34-285 Mobil: +49 160 3230 655 Mail: benjamin.bruns(a)cypp.de<mailto:benjamin.bruns@cypp.de> www.cypp.de<http://www.cypp.de/> | www.plathgroup.com<http://www.plathgroup.com/> Sitz der Gesellschaft: Hamburg HRB 132781 Amtsgericht Hamburg Ust-ID DE 298033875 Vertretungsberechtigter Geschäftsführerin: Maya von Holdt Dies ist eine vertrauliche Nachricht und nur für den Adressaten bestimmt. Es ist nicht erlaubt, diese Nachricht zu kopieren oder Dritten zugänglich zu machen. Sollten Sie diese Nachricht irrtümlich erhalten haben, benachrichtigen Sie den Versender bitte per E-Mail oder telefonisch und löschen Sie die Nachricht unverzüglich. This message is strictly confidential and intended solely for the use of the addressee. It is not allowed to copy or disseminate this message. Please notify the sender by e-mail or telephone if you have received this message by mistake and delete this message immediately. Von: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu> Im Auftrag von Laszlo Szemere (lszemere) Gesendet: Freitag, 28. August 2020 13:41 An: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Betreff: Re: [syslog-ng] syslog-ng is ignoring a network source Hello Ben, nc -u -l -p 514 -u : use UDP -l : listen in server mode, instead of sending -p : defines port number (If you need, you can also specify the local IP address to bind to with the -s option.) Br, Laci ________________________________ From: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu<mailto:syslog-ng-bounces@lists.balabit.hu>> on behalf of Bruns, Benjamin <Benjamin.Bruns(a)cypp.de<mailto:Benjamin.Bruns@cypp.de>> Sent: Friday, August 28, 2020 13:29 To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>> Subject: Re: [syslog-ng] syslog-ng is ignoring a network source CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Hello Laci, how do I check that by using netcat? Cheers, Ben Von: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu<mailto:syslog-ng-bounces@lists.balabit.hu>> Im Auftrag von Laszlo Szemere (lszemere) Gesendet: Freitag, 28. August 2020 12:21 An: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>> Betreff: Re: [syslog-ng] syslog-ng is ignoring a network source Hello Ben, our experience shows that having logs in Wireshark doesn't necessary means that they reach the applications. (Wireshark captures packets on the interface level.) So as a first step I would recommend to clarify that the logs from the second host indeed reaches Syslog-ng. (For UDP logs netcat usually is enough.) Once it is clear that those logs reaches the application level, we can focus on debugging Syslog-ng. By starting Syslog-ng with the following options, it would be much easier to examine the flow of messages: syslog-ng -Fdevt -F : start it in the foreground -d : debug mode -e : log messages to stderr -v : increases verbository -t : also enable trace messages Note: With these options enabled, Syslog-ng will produce a LOT of messages. So if you can turn off other logging sources temporary, than it will be much more easier to read those logs. At this point you should start to receive this kind of debug messages, which will indicate that Syslog-ng received the log messages from your host: [2020-08-28T10:09:43.289660] Incoming log entry; line='hello world' >From this point the easiest way is to start with a minimal config, and build up your final configuration step by step. Checking incoming logs in each steps. Br, Laci ________________________________ From: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu<mailto:syslog-ng-bounces@lists.balabit.hu>> on behalf of Bruns, Benjamin <Benjamin.Bruns(a)cypp.de<mailto:Benjamin.Bruns@cypp.de>> Sent: Friday, August 28, 2020 11:53 To: syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu> <syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>> Subject: [syslog-ng] syslog-ng is ignoring a network source CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Hello! I try to send syslogs from two firewalls to my syslog-ng host. The first one worked immediately, but logs of the second firewall seems to be ignored. Both syslogs come in via UDP on port 514 and I can see them in Wireshark on my syslog-ng host, but they disappear for my second firewall in a black hole. Both have Logstash as their destination configured. Any ideas? Thanks in advance! Cheers, Ben

2 3

Re: [syslog-ng] syslog-ng is ignoring a network source
by Bruns, Benjamin 01 Sep '20

01 Sep '20

Hello Laci, Question: I can successfully read the logs of the firewall that is located in the same network that my logserver is, but the second firewall is sending the logs from a different network segment to the log server. Does this causes the problem? Again: I can see the traffic on my network interface, but netcat doesn't receive anything on port 514 (for the second firewall). Does iptables fools me somehow? Mit freundlichen Grüßen / Best regards Benjamin Bruns IT Security Manager CYPP GmbH Gotenstrasse 15 20097 Hamburg Germany Telefon: +49 40 237 34-285 Mobil: +49 160 3230 655 Mail: benjamin.bruns(a)cypp.de<mailto:benjamin.bruns@cypp.de> www.cypp.de<http://www.cypp.de/> | www.plathgroup.com<http://www.plathgroup.com/> Sitz der Gesellschaft: Hamburg HRB 132781 Amtsgericht Hamburg Ust-ID DE 298033875 Vertretungsberechtigter Geschäftsführerin: Maya von Holdt Dies ist eine vertrauliche Nachricht und nur für den Adressaten bestimmt. Es ist nicht erlaubt, diese Nachricht zu kopieren oder Dritten zugänglich zu machen. Sollten Sie diese Nachricht irrtümlich erhalten haben, benachrichtigen Sie den Versender bitte per E-Mail oder telefonisch und löschen Sie die Nachricht unverzüglich. This message is strictly confidential and intended solely for the use of the addressee. It is not allowed to copy or disseminate this message. Please notify the sender by e-mail or telephone if you have received this message by mistake and delete this message immediately. Von: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu> Im Auftrag von Laszlo Szemere (lszemere) Gesendet: Freitag, 28. August 2020 13:41 An: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Betreff: Re: [syslog-ng] syslog-ng is ignoring a network source Hello Ben, nc -u -l -p 514 -u : use UDP -l : listen in server mode, instead of sending -p : defines port number (If you need, you can also specify the local IP address to bind to with the -s option.) Br, Laci ________________________________ From: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu<mailto:syslog-ng-bounces@lists.balabit.hu>> on behalf of Bruns, Benjamin <Benjamin.Bruns(a)cypp.de<mailto:Benjamin.Bruns@cypp.de>> Sent: Friday, August 28, 2020 13:29 To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>> Subject: Re: [syslog-ng] syslog-ng is ignoring a network source CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Hello Laci, how do I check that by using netcat? Cheers, Ben Von: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu<mailto:syslog-ng-bounces@lists.balabit.hu>> Im Auftrag von Laszlo Szemere (lszemere) Gesendet: Freitag, 28. August 2020 12:21 An: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>> Betreff: Re: [syslog-ng] syslog-ng is ignoring a network source Hello Ben, our experience shows that having logs in Wireshark doesn't necessary means that they reach the applications. (Wireshark captures packets on the interface level.) So as a first step I would recommend to clarify that the logs from the second host indeed reaches Syslog-ng. (For UDP logs netcat usually is enough.) Once it is clear that those logs reaches the application level, we can focus on debugging Syslog-ng. By starting Syslog-ng with the following options, it would be much easier to examine the flow of messages: syslog-ng -Fdevt -F : start it in the foreground -d : debug mode -e : log messages to stderr -v : increases verbository -t : also enable trace messages Note: With these options enabled, Syslog-ng will produce a LOT of messages. So if you can turn off other logging sources temporary, than it will be much more easier to read those logs. At this point you should start to receive this kind of debug messages, which will indicate that Syslog-ng received the log messages from your host: [2020-08-28T10:09:43.289660] Incoming log entry; line='hello world' >From this point the easiest way is to start with a minimal config, and build up your final configuration step by step. Checking incoming logs in each steps. Br, Laci ________________________________ From: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu<mailto:syslog-ng-bounces@lists.balabit.hu>> on behalf of Bruns, Benjamin <Benjamin.Bruns(a)cypp.de<mailto:Benjamin.Bruns@cypp.de>> Sent: Friday, August 28, 2020 11:53 To: syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu> <syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>> Subject: [syslog-ng] syslog-ng is ignoring a network source CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Hello! I try to send syslogs from two firewalls to my syslog-ng host. The first one worked immediately, but logs of the second firewall seems to be ignored. Both syslogs come in via UDP on port 514 and I can see them in Wireshark on my syslog-ng host, but they disappear for my second firewall in a black hole. Both have Logstash as their destination configured. Any ideas? Thanks in advance! Cheers, Ben

2 1

message being consistently dropped
by Wilson, Jonathan 01 Sep '20

01 Sep '20

We are running syslog-ng, open source edition, version 3.22.1, on RedHat Enterprise version 7.8. Among many other data sources, we receive syslog data from an F5 device that acts as a VPN server. One type of message that it sends us is consistently not making it into the log (though everything else, to my knowledge, does.) I wonder if anyone sees something wrong with the message that would prevent its being parsed/stored. The message (reconstructed from a packet capture, identification fields redacted or changed): <141>1 2020-08-04T08:10:58.769127-05:00 <fqdn here> apmd 12374 01490113:5: [F5@12276 hostname="fqdn" errdefs_msgno="01490113:5:" partition_name="Common" session_id="1c95e1e7" Access_Profile="/Common/blah" Partition="Common" Session_Id="1c95e1e6" Session_Variable_Name="session.machine_info.last.net_adapter.list.[0].mac_address" Session_Variable_Value="3C:D9:2B:33:9A:8E"] /Common/<blah>:Common:1c95e1e6: session.machine_info.last.net_adapter.list.[0].mac_address is 3C:D9:2B:33:9A:8E I inserted the line breaks to help Outlook. Note that although the message appears to be RFC5424-compliant I have discovered that it does not have whatever magic headers let syslog-ng handle it as structured data transparently. As a result, I cannot use the "syslog" transport but have to use the generic "network" transport and then use flags "syslog-protocol". This works OK for everything else from this source. The message doesn't appear in our syslog logs at all. It's consistent. The relevant parts of the syslog-ng configuration look like: options { ts_format(iso); use_dns(yes); dns_cache(1000); dns_cache_expire(30); use_fqdn(yes); keep_hostname(no); chain-hostnames(no); keep_timestamp(yes); stats_level(1); stats_freq(3600); normalize_hostnames(yes); log_fifo_size(10000); flush_lines(1000); log-msg-size(65536); trim-large-messages(yes); threaded(yes); time_reap(10); create_dirs(yes); dir_group(esmapp); dir_owner(esmapp); group(esmapp); owner(esmapp); perm(0644); }; source s_F5VPN_source { network( transport("tcp") ip(0.0.0.0) port(11000) keep-timestamp(yes) flags(no-multi-line,syslog-protocol) max-connections(50) log_iw_size(10000) log-fetch-limit(20) ); }; filter f_F5VPN_client { netmask(<blah>) and facility(local1); }; destination d_F5VPN_client { file( "/home/syslog/F5VPN_client.log" ts_format(rfc3164) ); }; log { source(s_F5VPN_source); filter(f_F5VPN_client); destination(d_F5VPN_client); flags(flow-control); }; Regards, Jon Jon Wilson | Principal System Engineer, IT Service Management | Information Technology | Vanderbilt University Medical Center jonathan.wilson(a)vumc.org<mailto:jonathan.wilson@vumc.org> | phone: 615-440-7895 | fax: 615-323-2181

2 1

No remote logging with hostname which has address is IPv6
by Alexandre Santos 28 Aug '20

28 Aug '20

Hi, I have configuration, which is sending log messages to a remote hostname (dns6server). The particularity of this remote server is that it is resolved to an IPv6 address. ping works fine: root@localhost:~# ping dns6server -c 3 PING dns6server(2620:38:4::8:4000:238) 56 data bytes 64 bytes from 2620:38:4::8:4000:238: icmp_seq=1 ttl=63 time=0.354 ms 64 bytes from 2620:38:4::8:4000:238: icmp_seq=2 ttl=63 time=0.264 ms 64 bytes from 2620:38:4::8:4000:238: icmp_seq=3 ttl=63 time=0.310 ms But syslog-ng seems not to be able to resolve the ipv6 address: Aug 27 20:58:37 localhost systemd[1]: mgmt-syslog-ng.service: Succeeded. Aug 27 20:58:37 localhost systemd[1]: Stopped External Logger Daemon. Aug 27 20:58:37 localhost systemd[1]: Starting External Logger Daemon... Aug 27 20:58:37 localhost ip[32709]: [2020-08-27T20:58:37.841389] WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; Aug 27 20:58:46 localhost ip[32709]: [2020-08-27T20:58:46.854714] Error resolving hostname; host='dns6server' Aug 27 20:58:46 localhost ip[32709]: [2020-08-27T20:58:46.854758] Initiating connection failed, reconnecting; time_reopen='60' Aug 27 20:58:46 localhost systemd[1]: Started External Logger Daemon. Any Ideas? I am sending the configuration and debug log in attachment. Thanks in Advance, Alex

2 5

syslog-ng 3.29 Debian/Ubuntu packages (OBS)
by Laszlo Budai 28 Aug '20

28 Aug '20

Hi, syslog-ng 3.29.1[1] packages are available from OBS repositories[2, 3, 4]. List of supported OSs: * Debian 8.0 * Debian 9.0 * Debian 10.0 * Raspbian 9.0 * Raspbian 10.0 * Ubuntu 14.04 * Ubuntu 16.04 * Ubuntu 18.04 * Ubuntu 18.10 * Ubuntu 19.04 * Ubuntu 19.10 * Ubuntu 20.04 Install ------- example: Debian 9.0 1. get release key wget -qO - http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… | sudo apt-key add - 2. add repo to APT sources eg.: /etc/apt/sources.list.d/syslog-ng-obs.list deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… ./ Then `apt-get update` and `apt-geat install syslog-ng-core` Links -------- [1] https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.29.1 [2] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… [3] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… [4] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… regards, Laszlo Budai

1 0

syslog-ng-3.29.1
by Laszlo Budai 28 Aug '20

28 Aug '20

3.29.1 Highlights * panos-parser(): parse Palo Alto PAN-OS logs Example: @include "scl.conf" log { source { network(transport("udp")); }; parser { panos-parser(); }; destination { elasticsearch-http( index("syslog-ng-${YEAR}-${MONTH}-${DAY}") type("") url("http://localhost:9200/_bulk") template("$(format-json --scope rfc5424 --scope dot-nv-pairs --rekey .* --shift 1 --exclude *future_* --exclude *dg_hier_level_* --scope nv-pairs --exclude DATE --key ISODATE @timestamp=${ISODATE})") ); }; }; (#3234<https://github.com/syslog-ng/syslog-ng/pull/3234>) Features * snmptrap: improve error message when missing dependency (#3363<https://github.com/syslog-ng/syslog-ng/pull/3363>) * disk queue: reduce memory usage during load (#3352<https://github.com/syslog-ng/syslog-ng/pull/3352>) * Allow dupnames flag to be used in PCRE expressions, allowing duplicate names for named subpatterns as explained here: https://www.pcre.org/original/doc/html/pcrepattern.html#SEC16 . Example: filter f_filter1 { match("(?<FOOBAR>bar)|(?<FOOBAR>foo)" value(MSG) flags(store-matches, dupnames)); }; (#3381<https://github.com/syslog-ng/syslog-ng/pull/3381>) Bugfixes * filter/regex: if there was a named match (?..)? that is optional to match, the previose or the next named matches might not be saved as named match. (#3393<https://github.com/syslog-ng/syslog-ng/pull/3393>) * tls: Fixed a bug, where ecdh-curve-list() were not applied at client side. (#3356<https://github.com/syslog-ng/syslog-ng/pull/3356>) * scratch-buffers: fix global.scratch_buffers_bytes.queued counter bug This bug only affected the stats_counter value, not the actual memory usage (i.e. memory usage was fine before) (#3355<https://github.com/syslog-ng/syslog-ng/pull/3355>) * wsl: fix infinite loop during startup (#3340<https://github.com/syslog-ng/syslog-ng/pull/3340>) * openbsd: showing grammar debug info for openbsd too, when -y command line option is used (#3339<https://github.com/syslog-ng/syslog-ng/pull/3339>) * stats-query: speedup syslog-ng-ctl query get "*" command. An algorithmic error view made syslog-ng-ctl query get "*" very slow with large number of counters. (#3376<https://github.com/syslog-ng/syslog-ng/pull/3376>) * syslogformat: fixing crashing with small invalid formatted logs see example in #3328<https://github.com/syslog-ng/syslog-ng/issues/3328> (#3364<https://github.com/syslog-ng/syslog-ng/pull/3364>) * cfg: fix config reload crash via introducing on_config_inited in LogPipe (#3176<https://github.com/syslog-ng/syslog-ng/pull/3176>) * config: fix error reporting * Error reporting was fixed for lines longer than 1024 characters. * The location of the error was incorrectly reported in some cases. (#3383<https://github.com/syslog-ng/syslog-ng/pull/3383>) * disk queue: fix possible crash during load, and possible false positive corruption detection (#3342<https://github.com/syslog-ng/syslog-ng/pull/3342>) * db-parser, pdbtool, graphite-output: fix glib assertion error The assertion happened in these cases * dbparser database load * argument parsing in graphite-output * pdbtool merge commad Syslog-ng emitted a glib assertion warning in the cases above, even in successful executions. If G_DEBUG=fatal-warnings environment variable was used, the warning turned into a crash. (#3344<https://github.com/syslog-ng/syslog-ng/pull/3344>) * stats: fix stats-ctl query crash when trying to reset all the counters syslog-ng-ctl query get '*' --reset (#3361<https://github.com/syslog-ng/syslog-ng/pull/3361>) Packaging * RHEL 7 packaging: fix logrotate file conflict with rsyslog (#3324<https://github.com/syslog-ng/syslog-ng/pull/3324>) * Debian packaging: python3-nose was removed from package dependencies. Pytest will run Python related unittests (for modules/python/pylib/syslogng/debuggercli/tests/) instead of nose. (#3343<https://github.com/syslog-ng/syslog-ng/pull/3343>) Notes to developers * light: test for assertion errors in glib for each testcases (#3344<https://github.com/syslog-ng/syslog-ng/pull/3344>) * Fix signal handling when an external library/plugin sets SIG_IGN Previously, setting SIG_IGN in a plugin/library (for example, in a Python module) resulted in a crash. (#3338<https://github.com/syslog-ng/syslog-ng/pull/3338>) * func-test: removed logstore_reader check, which was never reached (#3236<https://github.com/syslog-ng/syslog-ng/pull/3236>) * plugin_skeleton_creator: fixing a compiler switch Wrong compiler switch used in plugin_skeleton_creator. This caused a compiler warning. The grammar debug info did not appear for that module, when -y command line option was used. (#3339<https://github.com/syslog-ng/syslog-ng/pull/3339>) * Light test framework: get_stats and get_query functions to DestinationDriver class Two new functions added to DestinationDriver class which can be used for getting the stats and query output of syslog-ng-ctl. (#3211<https://github.com/syslog-ng/syslog-ng/pull/3211>) Other changes * internal(): limit the size of internal()'s temporary queue The internal() source uses a temporary queue to buffer messages. >From now on, the queue has a maximum capacity, the log-fifo-size() option can be used to change the default limit (10000). This change prevents consuming all the available memory in special rare cases. (#3229<https://github.com/syslog-ng/syslog-ng/pull/3229>) * network plugins: better timer defaults for TCP keepalive >From now on, syslog-ng uses the following defaults for TCP keepalive: * tcp-keepalive-time(): 60 * tcp-keepalive-intvl(): 10 * tcp-keepalive-probes(): 6 Note: so-keepalive() is enabled by default. (#3357<https://github.com/syslog-ng/syslog-ng/pull/3357>) Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Andras Mitzki, Antal Nemes, Attila Szakacs, Balazs Scheidler, Christian Tramnitz, chunmeng, Gabor Nagy, Laszlo Budai, Laszlo Szemere, László Várady, MileK, Norbert Takacs, Peter Czanik, Péter Kókai, Terez Nemes.

1 0

Re: [syslog-ng] syslog-ng is ignoring a network source
by Bruns, Benjamin 28 Aug '20

28 Aug '20

Hello Laci, how do I check that by using netcat? Cheers, Ben Von: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu> Im Auftrag von Laszlo Szemere (lszemere) Gesendet: Freitag, 28. August 2020 12:21 An: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Betreff: Re: [syslog-ng] syslog-ng is ignoring a network source Hello Ben, our experience shows that having logs in Wireshark doesn't necessary means that they reach the applications. (Wireshark captures packets on the interface level.) So as a first step I would recommend to clarify that the logs from the second host indeed reaches Syslog-ng. (For UDP logs netcat usually is enough.) Once it is clear that those logs reaches the application level, we can focus on debugging Syslog-ng. By starting Syslog-ng with the following options, it would be much easier to examine the flow of messages: syslog-ng -Fdevt -F : start it in the foreground -d : debug mode -e : log messages to stderr -v : increases verbository -t : also enable trace messages Note: With these options enabled, Syslog-ng will produce a LOT of messages. So if you can turn off other logging sources temporary, than it will be much more easier to read those logs. At this point you should start to receive this kind of debug messages, which will indicate that Syslog-ng received the log messages from your host: [2020-08-28T10:09:43.289660] Incoming log entry; line='hello world' >From this point the easiest way is to start with a minimal config, and build up your final configuration step by step. Checking incoming logs in each steps. Br, Laci ________________________________ From: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu<mailto:syslog-ng-bounces@lists.balabit.hu>> on behalf of Bruns, Benjamin <Benjamin.Bruns(a)cypp.de<mailto:Benjamin.Bruns@cypp.de>> Sent: Friday, August 28, 2020 11:53 To: syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu> <syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>> Subject: [syslog-ng] syslog-ng is ignoring a network source CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. Hello! I try to send syslogs from two firewalls to my syslog-ng host. The first one worked immediately, but logs of the second firewall seems to be ignored. Both syslogs come in via UDP on port 514 and I can see them in Wireshark on my syslog-ng host, but they disappear for my second firewall in a black hole. Both have Logstash as their destination configured. Any ideas? Thanks in advance! Cheers, Ben

2 1

syslog-ng is ignoring a network source
by Bruns, Benjamin 28 Aug '20

28 Aug '20

Hello! I try to send syslogs from two firewalls to my syslog-ng host. The first one worked immediately, but logs of the second firewall seems to be ignored. Both syslogs come in via UDP on port 514 and I can see them in Wireshark on my syslog-ng host, but they disappear for my second firewall in a black hole. Both have Logstash as their destination configured. Any ideas? Thanks in advance! Cheers, Ben

2 1

Message Macros
by Faine, Mark R. (MSFC-IS40)[NICS] 27 Aug '20

27 Aug '20

Is it possible to break up the $MESSAGE macro into smaller macros that constitute the entire message? The reason I ask is that we tried this format in our destination: template("${S_ISODATE} ${MESSAGE}\n") However, we get two dates now, since I'm assuming one is part of the $MESSAGE macro, and we also lost the $HOST in our messages. We just want the output to be exactly like it would be without specifying a template but with an ISO date, followed by the hostname/ip, and the rest of the message excluding the date. Thanks, -Mark

3 3

iptables and syslog-ng
by Faine, Mark R. (MSFC-IS40)[NICS] 13 Aug '20

13 Aug '20

I'm sure there is probably an existing canonical set of rules to allow syslog-ng traffic through iptables but I can't find it. I'm sorry, since I know this is probably a common question. I've been trying to get it to work but I seem to be missing something. This is what I have: iptables -A INPUT -i em3 -p udp --dport 514 -m state --state NEW,ESTABLISHED -j ACCEPT iptables -A OUTPUT -o em3 -p udp --sport 514 -m state --state ESTABLISHED -j ACCEPT iptables -A INPUT -i em1 -p tcp --dport 514 -m state --state NEW,ESTABLISHED -j ACCEPT iptables -A INPUT -i em1 -p tcp --dport 10514 -m state --state NEW,ESTABLISHED -j ACCEPT iptables -A OUTPUT -o em1 -p tcp --sport 514 -m state --state ESTABLISHED -j ACCEPT iptables -A OUTPUT -o em1 -p tcp --sport 10514 -m state --state ESTABLISHED -j ACCEPT We receive logs on udp 514 and tcp 10514 and we forward all of these logs to another remote server on the same ports. I've not seen any evidence of it but I've been told that we may also receive logs on tcp 514. I appreciate any assistance. We usually rely on hardware firewalls but new security guidelines are also requiring software firewalls be configured. Thanks, -Mark

1 0

Re: [syslog-ng] syslog-ng multiple VRF
by Terry Slattery 10 Aug '20

10 Aug '20

Alex, The simplest solution is to configure the network to share routes between the two VRFs. Linux now supports VRF, however, my quick read on the subject suggests that its configuration is complex. Applications like syslog-ng are rarely VRF-aware. It would have to bind to two different interfaces, one in each VRF. -tcs On 7/22/20 8:00 AM, syslog-ng-request(a)lists.balabit.hu wrote: > Date: Tue, 21 Jul 2020 15:18:46 +0100 > From: Alexandre Santos <alexandre.rosas.santos(a)gmail.com> > To: "Syslog-ng users' and developers' mailing list" > <syslog-ng(a)lists.balabit.hu> > Subject: [syslog-ng] syslog-ng multiple VRF > Message-ID: > <CAHxG_72hhkfCBu3j9nPxr_MVwKevJtFcpcRhVJLYPPYx4Mys1g(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > Hi All, > > I have a question regarding syslog-ng and VRF. > I want to read from a syslog source, which interface is in the default VRF, > and send the logs to a syslog/network destination interface which is in a > MGMT VRF. > > Can syslog-ng support this? > If yes, what are the aspects I should be careful about? > > Thanks and regards, > Alex -- Terry Slattery CCIE #1026 NetCraftsmen www.netcraftsmen.com

5 7

Kafka performance
by Ronald Fenner 07 Aug '20

07 Aug '20

I'm trying to figure out why the performance of syslog-ng sending to kafka has tanked considerably. Everything was running fine up until 7/30 and then all of sudden syslog-ng is having problems pushing messages to kafka at a quick pace. I've updated to 3.28.1 and it's still having problems. I was using disk buffering but I removed it an d doesn't seem to be the bottleneck. I feel it's something with syslog ng as I can use the kafka perf test to push messages at a very quick rate to the cluster. I was able to push 1 million 1k test messages in several seconds. Syslog-ng seems to be able to only push a couple of messages a second. Any thoughts on what could be tanking it's performance? It's running on Amazon Linux 2. I've rebuilt our base image with the latest version so we can switch between kafka-c and the java version however since the kafka-c topic can't be templated at the moment until a pull request is released I can't switch to it to see if it fairs better. Ronald Fenner Network Architect Game Circus LLC. rfenner(a)gamecircus.com

2 3

Sylog-ng and F5 Load balancer.
by Peter Griggs 07 Aug '20

07 Aug '20

Hello Does anyone have any experience of syslog-ng behind F5 load balancer and preserving source IP address? The F5 can put the X-Forwarding header or TCP Options value but I don't believe syslog-ng can understand either of these - Am I wrong? If I am then I am assuming I can have a filter that overwrites $HOST with the value of X-Forwarding or TCP Options. Any help appreciated. Thanks Peter.

3 7

EWMM and sudo app-parser
by Fabien Wernli 06 Aug '20

06 Aug '20

Hi, I'm investigating using the EWMM forwarding model. Consider the following setup: Linux hosts collect logs using `system()` send them over using `syslog-ng()` destination to a remote host that collects them using `default-network-drivers()` source. It seems to me that the sudo app parsing is fired up twice: 1. On the sender side because `system()` expands to something including the `sudo-parser()` SCL 2. On the receiver side because `default-network-drivers()` expands to something involving the `app-parser()` This happens also when using `syslog()` source on the sender side, which is why I noticed this behaviour. So my question is, is there something wrong with that model ?

2 4

Kafka-c topics dynamic
by Ronald Fenner 06 Aug '20

06 Aug '20

When I was using the java implementation I extracted the topic dynamically from the message to route the message to the proper topic. I see in Kafka c implementation that the topic can't use templates so my old method of doing this topic("${topic}.${environment}") No longer works. Is there a way to reproduce this or do I need to go back to the java version for now? Ronald Fenner Network Architect Game Circus LLC. rfenner(a)gamecircus.com

2 2

journald prefix
by Fabien Wernli 05 Aug '20

05 Aug '20

Hi, I'm in the process of simplifying my structured logging config. Most of my systems now have journald shipping local logs to syslog-ng via the `system()` auto-expanding source. While this is convenient, as it makes it possible to use the same config for all my systems (systemd and rest of world), it also makes it impossible to configure the underlying driver's defaults. The task at hand is shipping the journald additional fields, which would be easy to do using the following excerpt from the documentation: source s_journald { systemd-journal(prefix(".SDATA.journald.")); }; This is not possible of course using `system()`. I've got a couple of ideas on how to handle this, but I'd like to hear other (probably less insane than my own) ways first :-) Cheers

2 2

Error trying to use kafka-c
by Ronald Fenner 05 Aug '20

05 Aug '20

I'm trying to switch from the java version of kafka to the c version and have built syslog with the following config: ./configure --prefix=/opt/syslog-ng --enable-json --enable-python --with-python=2.7 --enable-kafka However when I try syntax check the config it's returning this error: Error parsing destination statement, destination plugin kafka-c not found in block destination kafka() at /opt/syslog-ng/share/syslog-ng/include/scl/kafka/kafka.conf:24:4:5-4:12: 1 2 #Start Block block destination kafka() at /opt/syslog-ng/share/syslog-ng/include/scl/kafka/kafka.conf:24 3 4-----> kafka-c(bootstrap_servers("kafka.service.consul:9092") topic("syslog-ng-production") ); 4-----> ^^^^^^^ 5 6 #End Block block destination kafka() at /opt/syslog-ng/share/syslog-ng/include/scl/kafka/kafka.conf:24 7 Included from /opt/syslog-ng/etc/syslog-ng.conf:33:5-36:6: 28 source s_rtl_raw_backup { 29 file("/var/gamecircus/log/rtl-raw-backup.log" flags(no-parse) log_msg_size(1000000)); 30 }; 31 32 destination d_kafka_unstructured { 33----> kafka ( 33----> ^^^^^^^^ 34----> bootstrap-servers("kafka.service.consul:9092") 35----> topic("syslog-ng-`DeployedEnvironment`") 36----> ); 37 }; 38 The top of my config looks like this @version: 3.28 @include "scl.conf" @include "conf.d/gc-parsers.conf" @define kafka-implementation kafka-c Everything looks correct just not sure why it's having a hard time finding it. Ronald Fenner Network Architect Game Circus LLC. rfenner(a)gamecircus.com

2 1

Deadlock with full disk buffer
by Ronald Fenner 04 Aug '20

04 Aug '20

I've run into a problem where we had enabled reliable disk buffering on our Kafka destination and it's disk buffer eventually filled the drive it's stored on party because the drive is sized to the buffer it's holding. When restarting syslog-ng it complains with "Daemon exited due to a deadlock/signal/failure, restarting; exitcode='6'" The version we are using is 3.23.1 I'm not sure why the disk buffer filled up the only thing I can think of is that syslog-ng was unable to send message to the Kafka destination, however our Kafka cluster is up and function fine and I can even consume messages from a topic that we are pushing messages to on one of the instances affected. A question once a reliable message is sent is it removed from the buffer, i assume that's the case. The other question is it possible the java Kafka plugin can't keep up with the amount of mesages we are sending though syslog-ng. We see 12.5 million requests per day to the servers with the disk buffers. Each request generates at least 2 messages, one access log message, one application log so that's roughly 25 million messages a day. I designed the disk buffers to hold roughly 2 days worth of data if for some reason the Kafka cluster went down which in this case doesn't seem to be the case since monitoring tools show it's online and none of our hourly consumers complained about being able to connect. I'll have to build a recovery server for the buffers since they are housed on attached drives the drive are preserved when the instance is terminated this was done to make sure we didn't lose any messages not sent when the autoscaling scaled out a instance. I'm currently manually terminating the affected instances to keep those buffer and restore out logging. Once I have it, if more info is needed I can see what happens with trying to start syslog-ng up when it's also not receiving a bunch of messages as well. Ronald Fenner Network Architect Game Circus LLC. rfenner(a)gamecircus.com

2 3

Error message in syslog-ng (LOGMSG_REFCACHE_VALUE_TO_REF(old_value) >= 1)
by Ahmed Soyef 29 Jul '20

29 Jul '20

Hi All, We're seeing some instances of this error message ERROR:lib/logmsg/logmsg.c:1521:log_msg_ref: assertion failed: (LOGMSG_REFCACHE_VALUE_TO_REF(old_value) >= 1) I've had a quick search about and can't figure out what it actually means. Can anyone shed some light and possibly some guidance on how to avoid it happening? Regards, Soyef ********************************************************************** This email and any files transmitted with it are private and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please return it to the address it came from telling them it is not for you and then delete it from your system. This email message has been swept for computer viruses. **********************************************************************

2 1

File destination is not being written, when it can not bind remote host address
by Alexandre Santos 29 Jul '20

29 Jul '20

Hi, I do not know if this a seen issue or not. I am using syslog-ng version 3.19 on a Debian 10. I have the configuration which is attached in this email: syslog-ng.conf, that has one local file destination and one remote host destination. 1. Start the syslog-ng The localip of syslog destination is not available and therefore I have these errors, as expected: syslog-ng[1482]: [2020-07-27T12:58:00.069422] Error binding socket; addr='AF_INET(10.0.2.6:0)', error='Cannot assign requested address (99)' syslog-ng[1482]: [2020-07-27T12:58:00.069439] Initiating connection failed, reconnecting; time_reopen='60' 2. After some time running syslog-ng stops sending the logs to the local file. Why? 3. If I reload the syslog-ng configuration, without the remote destination, the old logs are immediately flushed to the local file. Jul 27 *15:05:55* localhost syslog-ng[29680]: [*2020-07-27T15:05:55.378*019] Incoming log entry from journal; message='[*2020-07-27T13:44:06.890*952] Outgoing message; message=\'<30>1 2020-07-27T13:44:06.772+00:00 localhost syslog-ng 29680 - - [2020-07-27T13:44:06.772722] Processing the time zone file (32bit part); filename=\\'/usr/share/zoneinfo/UTC\\'\x0a\'' Jul 27 15:05:55 localhost syslog-ng[29680]: [2020-07-27T15:05:55.378032] json-parser(): no marker at the beginning of the message, skipping JSON parsing ; input='[2020-07-27T13:44:06.890952] Outgoing message; message=\'<30>1 2020-07-27T13:44:06.772+00:00 localhost syslog-ng 29680 - - [2020-07-27T13:44:06.772722] Processing the time zone file (32bit part); filename=\\'/usr/share/zoneinfo/UTC\\'\x0a\'', marker='@cim:' Is this a new issue? Is there any configuration to prevent this? In attachment also the journalctl of syslog-ng in debug mode (syslog-ng -Fvde). Thanks and regards, Alex

2 1

setting sequenceId in forwarded log messages read from journald reader
by Peter Vollmer 29 Jul '20

29 Jul '20

Hi, I am currently trying to find a way to set meta.sequenceId of log messages that have been read from the locally running systemd-journal to forward them to a remote syslog server that expects the logs to contain a sequenceId according to RFC 5424 section 7.3.1. I found that a sequence number could be taken from the __CURSOR field "i=..." of the journald log: # journalctl -o json-pretty -f ... "__CURSOR" : "s=02a7b30ba17b4a43846f265706bd3a70;i=f01;b=ba633698f20848e480bca4e72476e4d3;m=1a355c1d5;t=5ab670340c8ea;x=33389988ef680e7e", ... My problem is that the journal reader does not seem to parse the __CURSOR string when reading from journald logs. Is there a way to get this information into meta.sequenceId of the forwarded log without modifying the systemd-journal module in syslog-ng ? Thank you for any ideas and best regards Peter Vollmer

3 2

Certificate ip authentication
by Alexandre Damas 27 Jul '20

27 Jul '20

Hi, As my objective on the utilization of syslog-ng is the authentication and certification of received security and auditing events, I implemented an internal CA which generates a certificate per syslog-ng client. Once there was someone that misconfigured a client and got a certificate for a different client. The funny part of it is that no one observed any problem as the certificate, which was generated for a client having an ip (different from the one configured on the machine), was working and communications (using ALTP with TLS) went up for the exchange of messages. Has anyone experienced this? Does someone have a clue on how to prevent certificate reutilization on the client side? On Linux side I did not find any way of preventing the utilization of a certificate for one machine, that was issued for a different machine, having a different ip. If there is no cross checking on the server for the ip addresses on the certificate with the ip address on the received event, I don't have a way of non-repudiate a received event and the client can reuse the certificate for other machines, allowing the events to be received on the server. Kind Regards Alexandre Damas

2 2

Configuration validation Issue with snmptrap module
by Luca Da Col 23 Jul '20

23 Jul '20

Running syslog-ng 3.27.1 on Centos 7. Configuration validation returns an error with snmptrap command. Configuration example has been taken directly from the OSE documentation. Error error was reproducible with version available on epel repository (3.5), though documentation indicates the snmptrap is available from 3.10 Configuration file includes statements: @version:3.22 @include "scl.conf" Any assistance is appreciated, Thank you, Luca [root@adev syslog-ng]# syslog-ng --syntax-only Error parsing source statement, source plugin snmptrap not found in /etc/syslog-ng/syslog-ng.conf:92:5-92:13: 90 log { 91 source { 92----> snmptrap(filename("/var/log/snmptrapd.log")); 92----> ^^^^^^^^ 93 }; 94 destination { 95 file("/var/log/example.log"); 96 }; 97 }; [root@dev syslog-ng]# syslog-ng -V syslog-ng 3 (3.27.1) Config version: 3.22 Installer-Version: 3.27.1 Revision: Compile-Date: Jun 8 2020 07:54:17 Module-Directory: /usr/lib64/syslog-ng Module-Path: /usr/lib64/syslog-ng Include-Path: /usr/share/syslog-ng/include Available-Modules: affile,afprog,afsocket,afstomp,afuser,basicfuncs,confgen,cryptofuncs,csvparser,dbparser,linux-kmsg-format,syslogformat,system-source,add-contextual-data,appmodel,cef,disk-buffer,examples,graphite,hook-commands,json-plugin,kvformat,map-value-pairs,pseudofile,sdjournal,stardate,tags-parser,tfgetent,timestamp,xml Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-Linux-Caps: on Enable-Systemd: on [root@dev syslog-ng]#

2 1

syslog-ng multiple VRF
by Alexandre Santos 22 Jul '20

22 Jul '20

Hi All, I have a question regarding syslog-ng and VRF. I want to read from a syslog source, which interface is in the default VRF, and send the logs to a syslog/network destination interface which is in a MGMT VRF. Can syslog-ng support this? If yes, what are the aspects I should be careful about? Thanks and regards, Alex

2 2

syslog-ng restarts due to interface not ready
by Alexandre Santos 21 Jul '20

21 Jul '20

Hi, I have a configuration where one of my sources is a syslog source: syslog(ip(127.1.1.1) transport("udp") port(514) keep-alive(no)); But when the system starts up, the interface is not ready yet, and I get the following errors: Jun 26 09:17:50 localhost systemd[1]: Starting System Logger Daemon... Jun 26 09:17:52 localhost syslog-ng[2301]: [2020-06-26T09:17:52.485148] WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; Jun 26 09:17:52 localhost syslog-ng[2301]: [2020-06-26T09:17:52.487159] Current position not matches to the saved cursor position, seek to head; cursor='s=f62c0e80f50948deb29cf2d28230a Jun 26 09:17:52 localhost syslog-ng[2301]: [2020-06-26T09:17:52.487867] Error resolving hostname; host='127.1.1.1' Jun 26 09:17:52 localhost syslog-ng[2301]: [2020-06-26T09:17:52.487878] Error initializing message pipeline; plugin_name='syslog', location='/etc/syslog-ng/syslog-ng.conf:43:5' Jun 26 09:17:52 localhost systemd[1]: syslog-ng.service: Main process exited, code=exited, status=2/INVALIDARGUMENT Jun 26 09:17:52 localhost systemd[1]: syslog-ng.service: Failed with result 'exit-code'. Jun 26 09:17:52 localhost systemd[1]: Failed to start System Logger Daemon. Jun 26 09:17:52 localhost systemd[1]: syslog-ng.service: Service RestartSec=100ms expired, scheduling restart. Jun 26 09:17:52 localhost systemd[1]: syslog-ng.service: Scheduled restart job, restart counter is at 1. When the systemd reaches the 5 attempts it disables the syslog-ng.service. Do you know some way to fix this, by adding something, the syslog-ng configuration? Thanks in Advance, Alex

4 4

disk buffer files permissions
by Pal, Laszlo 13 Jul '20

13 Jul '20

Hi, We want to monitor disk-buffer files, so we can detect if there is are some issues with the destinations. syslog-ng creates these files with permissions like this otal 360 -rw-------. 1 root root 4096 Oct 10 2019 syslog-ng-00000.qf -rw-------. 1 root root 4096 Jul 12 03:49 syslog-ng-00001.qf So, our monitor script (for zabbix) cannot stat files because it is running under user zabbix. I'm not sure how to submit a feature request nowadays, but it would be great of perm() option added to disk-buffer() Any chance? Thx L:

3 4

Insider 2020-07: TLS; capabilities; 3.27;
by Peter Czanik (pczanik) 09 Jul '20

09 Jul '20

Dear syslog-ng users, This is the 83rd issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Simplifying CA handling in syslog-ng TLS connections ---------------------------------------------------- When talking to users about the TLS-encrypted message transfer, almost everyone immediately complains about configuring a certificate authority (CA) in syslog-ng. You needed to create a hash and create a symbolic link to the CA file based on the hash. Not anymore. While this old method is still available, there is now a much easier way: the new ca-file() option. https://www.syslog-ng.com/community/b/blog/posts/simplifying-ca-handling-in… Working around Linux capabilities problems for syslog-ng -------------------------------------------------------- No, SELinux is not the cause of all permission troubles on Linux. For example, syslog-ng makes use of the capabilities system on Linux to drop as many privileges as possible, as early as possible. But it might cause problems in some corner cases, as even when running as root, syslog-ng cannot read files owned by a different user. Learn from this blog how you can figure out if you have a SELinux or capabilities problem and how to fix it if you do. https://www.syslog-ng.com/community/b/blog/posts/working-around-linux-capab… Figuring out where a message arrived, and other syslog-ng 3.27 tricks ---------------------- Version 3.27 of syslog-ng has brought many smaller, but useful features to us. The new Sumo Logic destination was already covered in an earlier blog. You can now also check exactly where a message arrived on a network source (IP address, port and protocol). Rewriting the facility of a syslog message was also made easy. For a complete list of new features and changes, check the release notes at https://github.com/syslog-ng/syslog-ng/releases/tag/syslog-ng-3.27.1 You can learn more about these features at https://www.syslog-ng.com/community/b/blog/posts/figuring-out-where-a-messa… NEW RELEASES * Version 3.28.1 released: https://github.com/syslog-ng/syslog-ng/releases/tag/syslog-ng-3.28.1 * syslog-ng PE version 7.0.21 is coming soon with MSSQL and Oracle SQL sources, and you can send logs to Google Pub/Sub. For details, check the documentation at https://support.oneidentity.com/syslog-ng-premium-edition/7.0.21 WEBINARS * Learn how to do custom HTTP REST API Authentication with syslog-ng: https://www.syslog-ng.com/event/customize-http-rest-apis-authentication-wit… * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Convert logs back into syslog-ng logs
by Faine, Mark R. (MSFC-IS40)[NICS] 07 Jul '20

07 Jul '20

I would like to read lines from a file back into log data that syslog-ng can parse. What would be involved in doing this? Thanks, -Mark

2 3

syslog-ng 3.28.1 RPM packages
by Peter Czanik (pczanik) 24 Jun '20

24 Jun '20

Hi, As usual, my unofficial syslog-ng rpm packages for openSUSE/SLES and Fedora/RHEL/CentOS first appear in my git snapshot repositories. I hope to create the usual dedicated repositories later this week and that will be followed by a separate announcement. For now you can install/test 3.28.1 following instructions from https://syslog-ng.com/blog/rpm-packages-from-syslog-ng-git-head/ Any feedback is appreciated! Here is a list of changes since the last release: * Java modules (HDFS, Elasticsearch, Kafka, etc.) removed. The Java binding itself stays, where JDK 1.8 is available. * Logrotate script on Fedora is renamed to syslog-ng and only installed if rsyslog is not on the system * Logrotate script on RHEL 8 is only installed if rsyslog is not on the system to avoid conflict * * Logrotate bug on RHEL 7 fixed (the same logrotate script is used as for rsyslog to avoid conflict * Bye, CzP Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

syslog-ng 3.28 Debian/Ubuntu packages (OBS)
by Laszlo Budai 23 Jun '20

23 Jun '20

Hi, syslog-ng 3.28.1[1] packages are available from OBS repositories[2, 3, 4]. List of supported OSs: * Debian 8.0 * Debian 9.0 * Debian 10.0 * Raspbian 9.0 * Raspbian 10.0 * Ubuntu 14.04 * Ubuntu 16.04 * Ubuntu 18.04 * Ubuntu 18.10 * Ubuntu 19.04 * Ubuntu 19.10 * Ubuntu 20.04 Install ------- example: Debian 9.0 1. get release key wget -qO - http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… | sudo apt-key add - 2. add repo to APT sources eg.: /etc/apt/sources.list.d/syslog-ng-obs.list deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… ./ Then `apt-get update` and `apt-get install syslog-ng-core` Links -------- [1] https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.28.1 [2] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… [3] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… [4] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… regards, Laszlo Budai

1 0

syslog-ng-3.28.1
by Laszlo Budai 23 Jun '20

23 Jun '20

3.28.1 Highlights * http: add support for proxy option Example: log { source { system(); }; destination { http( url("SYSLOG_SERVER_IP:PORT") proxy("PROXY_IP:PORT") method("POST") ); }; }; (#3253<https://github.com/syslog-ng/syslog-ng/pull/3253>) Features * map: template function This template function applies a function to all elements of a list. For example: $(map $(+ 1 $_) 0,1,2) => 1,2,3. (#3301<https://github.com/syslog-ng/syslog-ng/pull/3301>) * use-syslogng-pid(): new option to all sources If set to yes, syslog-ng overwrites the message's ${PID} macro to its own PID. (#3323<https://github.com/syslog-ng/syslog-ng/pull/3323>) Bugfixes * affile: eliminate infinite loop in case of a spurious file path If the template evaluation of a log message will result to a spurious path in the file destination, syslog-ng refuses to create that file. However the problematic log message was left in the msg queue, so syslog-ng was trying to create that file again in time-reopen periods. >From now on syslog-ng will handle "permanent" file errors, and drop the relevant msg. (#3230<https://github.com/syslog-ng/syslog-ng/pull/3230>) * Fix minor memory leaks in error scenarios (#3265<https://github.com/syslog-ng/syslog-ng/pull/3265>) * crypto: fix hang on boot due to lack of entropy (#3271<https://github.com/syslog-ng/syslog-ng/pull/3271>) * Fix IPv4 UDP destinations on FreeBSD UDP-based destinations crashed when receiving the first message on FreeBSD due to a bug in destination IP extraction logic. (#3278<https://github.com/syslog-ng/syslog-ng/pull/3278>) * network sources: fix TLS connection closure RFC 5425 specifies that once the transport receiver gets close_notify from the transport sender, it MUST reply with a close_notify. The close_notify alert is now sent back correctly in case of TLS network sources. (#2811<https://github.com/syslog-ng/syslog-ng/pull/2811>) * disk-buffer: fixes possible crash, or fetching wrong value for logmsg nvpair (#3281<https://github.com/syslog-ng/syslog-ng/pull/3281>) * packaging/debian: fix mod-rdkafka Debian packaging (#3282<https://github.com/syslog-ng/syslog-ng/pull/3282>) * kafka destination: destination halts if consumer is down, and kafka's queue is filled (#3305<https://github.com/syslog-ng/syslog-ng/pull/3305>) * file-source: Throw error, when follow-freq() is set with a negative float number. (#3306<https://github.com/syslog-ng/syslog-ng/pull/3306>) * stats-freq: with high stats-freq syslog-ng emits stats immediately causing high memory and CPU usage (#3320<https://github.com/syslog-ng/syslog-ng/pull/3320>) * secure-logging: bug fixes (#3284<https://github.com/syslog-ng/syslog-ng/pull/3284>) * template arguments are now consistently checked * fixed errors when mac file not provided * fixed abort when derived key not provided * fixed crash with slogkey missing parameters * fixed secure-logging on 32-bit architectures * fixed CMake build Other changes * dbld: Fedora 32 support (#3315<https://github.com/syslog-ng/syslog-ng/pull/3315>) * dbld: Removed Ubuntu Eoan (#3313<https://github.com/syslog-ng/syslog-ng/pull/3313>) * secure-logging: improvements (#3284<https://github.com/syslog-ng/syslog-ng/pull/3284>) * removed 1500 message length limitation * slogimport has been renamed to slogencrypt * $(slog) will not start anymore when key is not found * internal messaging (warning, debug) improvements * improved memory handling and error information display * CMake build improvements * switched to GLib command line argument parsing * the output of slogkey -s is now parsable * manpage improvements Notes to developers * dbld: devshell is now upgraded to Ubuntu Focal (#3277<https://github.com/syslog-ng/syslog-ng/pull/3277>) * dbld/devshell: Multiple changes: * Added snmptrapd package. * Added support for both python2 and python3. (#3222<https://github.com/syslog-ng/syslog-ng/pull/3222>) * threaded-source: fully support default-priority() and default-facility() (#3304<https://github.com/syslog-ng/syslog-ng/pull/3304>) * CMake: fix libcap detection (#3294<https://github.com/syslog-ng/syslog-ng/pull/3294>) * Fix atomic_gssize_set() warning with new glib versions (#3286<https://github.com/syslog-ng/syslog-ng/pull/3286>) Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Airbus Commercial Aircraft, Andras Mitzki, Antal Nemes, Attila Szakacs, Balazs Scheidler, Gabor Nagy, Laszlo Budai, Laszlo Szemere, László Várady, Péter Kókai, Vatsal Sisodiya, Vivin Peris.

1 0

request for testing: logrotate scripts in Fedora/RHEL packages
by Peter Czanik (pczanik) 23 Jun '20

23 Jun '20

Hi, There were multiple tickets, bug reports about logrotate scripts in syslog-ng RPM packages for Fedora/RHEL. https://bugzilla.redhat.com/show_bug.cgi?id=1802165 https://github.com/syslog-ng/syslog-ng/issues/1384#issuecomment-644716350 I have now a possible solution available for testing in my git snapshot repo: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng-githead/ * Installs the same logrotate script on RHEL/CentOS 7 as rsyslog * On RHEL/CentOS 8 and Fedora the logrotate script is called syslog-ng and uses systemctl to reload syslog-ng after rotation, just as the rsyslog variant. It is in a separate subpackage, and installed only when rsyslog is not on the system (for reasons check the RH bugzilla ticket: in short it would not be used anyway due to conflicts) I tested it on Fedora 32 and CentOS 7, both with and without rsyslog on the system. It worked fine for me, but any feedback is appreciated. If it is tested to work by someone else than me, I'll push the changes both to the official EPEL package and to the syslog-ng github repo. Bye, CzP Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

4 7

syslog-ng rpm java changes
by Secure-Logging, Generic 22 Jun '20

22 Jun '20

Hi Peter, apparently, I missed the poll. But I can assure you that we are not using Java in the syslog-ng version that are shipped to our customers. We are even building our own packages for syslog-ng on RHEL 7 to get rid of Java and Python. The systems we deliver have a minimal version of syslog-ng installed and the only extra module we currently use is the secure logging module. Regards, Stephan The information in this e-mail is confidential. The contents may not be disclosed or used by anyone other than the addressee. Access to this e-mail by anyone else is unauthorised. If you are not the intended recipient, please notify Airbus immediately and delete this e-mail. Airbus cannot accept any responsibility for the accuracy or completeness of this e-mail as it has been sent over public networks. If you have any concerns over the content of this message or its Accuracy or Integrity, please contact Airbus immediately. All outgoing e-mails from Airbus are checked using regularly updated virus scanning software but you should take whatever measures you deem to be appropriate to ensure that this message and any attachments are virus free.

1 0

syslog-ng rpm java changes
by Peter Czanik (pczanik) 22 Jun '20

22 Jun '20

Hi, Over a week ago I ran a Twitter poll and also asked around here on the mailing list about if/how you use Java in syslog-ng. No responses here and only a few "no" responses on Twitter. We know about some users, who use the Java bindings, but not our modules. Any previous users of Java modules I asked directly told me, that they already changed to kafka-c and elasticsearch-http. A few days later Fedora announced, that they are removing JDK 1.8 support from Rawhide. So, here is a list of changes and plans for the official distro syslog-ng packages and for my unofficial packages: * removed Java support completely from the official Fedora Rawhide package (no support for JDK 1.11 yet in syslog-ng), to avoid dropping syslog-ng from the distro * my unofficial packages will follow, once JDK 1.8 is really removed from the distro (end of July) * keep Java bindings everywhere else, as long as 1.8 is available * removed java modules (drivers) from all of my unofficial packages (Java-based Elasticsearch, Kafka, HDFS) My syslog-ng git snapshot packages (https://syslog-ng.com/blog/rpm-packages-from-syslog-ng-git-head/) already have these changes, and my 3.28 packages will be derived from these, once the release is available. Bye, Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Some logs written, some are not
by Jason Brown 16 Jun '20

16 Jun '20

I’m hoping someone can help here, because I’m at my wits end. System info: syslog-ng OSE 3.27 OS: CentOS 7.8 (though, I get similar results on Ubuntu 18.04) I have some kubernetes pods sending logs to a remote syslog-ng server. I have an issue where some logs are not being created. These are specifically received via TCP. I can see the message inbound by doing a packet capture, but nothing is written. One of the messages that stands out is: "Error processing message <-1>" Here is my scenario: Start a deployment with 3 pods: Pod1: log created on syslog-ng server: /data/2020/06/09/pod1/logs/app.log Pod2: nothing Pod3: nothing Delete and deploy again: Pod1: log created on syslog-ng server: /data/2020/06/09/pod1/logs/app.log Pod2: log created on syslog-ng server: /data/2020/06/09/pod2/logs/app.log Pod3: nothing Delete and deploy again: Pod1: nothing Pod2: nothing Pod3: nothing There seems to be no pattern. The pods that log, do so successfully. All pods should be logging the same exact date except for timestamps and the pod name. A packet capture shows healthy traffic inbound, with no errors, even from the pods that are not logging. The only indication of an issue is the trace log. Here’s an example from one of the non-writing pods, with a message id “0x7f2af8003800” : Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Initial message parsing follows; Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='PROGRAM', value='testlogger', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='LEGACY_MSGHDR', value='testlogger: ', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='HOST', value='testlogger-service-54956569bb-kqsjs', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='MESSAGE', value='2020-06-09 11:23:47,295 (main) INFO [c.b.s.r.s.TestLoggerServer] [trkId=] Server starting in Staging environment ', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='HOST_FROM', value='10.13.143.181', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='SOURCE', value='s_tcp_net_514', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:147:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local7', location='/etc/syslog-ng/syslog-ng.conf:147:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='20', valid_fac='00800000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local7', location='/etc/syslog-ng/syslog-ng.conf:147:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:146:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local6', location='/etc/syslog-ng/syslog-ng.conf:146:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='20', valid_fac='00400000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local6', location='/etc/syslog-ng/syslog-ng.conf:146:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:145:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local5', location='/etc/syslog-ng/syslog-ng.conf:145:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='20', valid_fac='00200000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local5', location='/etc/syslog-ng/syslog-ng.conf:145:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:142:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local2', location='/etc/syslog-ng/syslog-ng.conf:142:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='20', valid_fac='00040000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local2', location='/etc/syslog-ng/syslog-ng.conf:142:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:141:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local1', location='/etc/syslog-ng/syslog-ng.conf:141:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='20', valid_fac='00020000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local1', location='/etc/syslog-ng/syslog-ng.conf:141:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:144:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local4', location='/etc/syslog-ng/syslog-ng.conf:144:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='20', valid_fac='00100000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: severity() evaluation started; pri='6', valid_pri='000000ff', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> rewrite rule evaluation begin; rule='r_newlines', location='/etc/syslog-ng/syslog-ng.conf:186:5', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Message was cloned; original_msg='0x7f2af8003800', new_msg='0x7f2af8004830' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Performing subst() rewrite failed, pattern did not match; rule='r_newlines', value='MESSAGE', input='2020-06-09 11:23:47,295 (main) INFO [c.b.s.r.s.TestLoggerServer] [trkId=] Server starting in Staging environment ', type='(null)', pattern='u2028', replacement='\x0a', location='/etc/syslog-ng/syslog-ng.conf:186:5’ Further down in the log, I see: Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Initial message parsing follows; Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='HOST', value='', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='MESSAGE', value='Error processing log message: <-1>Jun 9 11:23:47 testlogger-service-54956569bb-2c5wl testlogger: 2020-06-09 11:23:47,341 (main) INFO [c.b.s.r.s.TestLoggerServer] [trkId=] Server starting in Staging environment \x0a\x0a<-1>Jun 9 11:23:47 testlogger-service-54956569bb-2c5wl testlogger: 2020-06-09 11:23:47,346 (main) INFO [c.b.s.r.s.TestLoggerServer] [trkId=] Test log message \x0a\x0a>@<', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='PROGRAM', value='syslog-ng', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='PID', value='28873', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='HOST_FROM', value='10.13.98.167', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='HOST', value='10.13.98.167/10.13.98.167', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='SOURCE', value='s_tcp_net_514', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:147:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local7', location='/etc/syslog-ng/syslog-ng.conf:147:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='5', valid_fac='00800000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local7', location='/etc/syslog-ng/syslog-ng.conf:147:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:146:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local6', location='/etc/syslog-ng/syslog-ng.conf:146:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='5', valid_fac='00400000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local6', location='/etc/syslog-ng/syslog-ng.conf:146:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:145:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local5', location='/etc/syslog-ng/syslog-ng.conf:145:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='5', valid_fac='00200000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local5', location='/etc/syslog-ng/syslog-ng.conf:145:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:142:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local2', location='/etc/syslog-ng/syslog-ng.conf:142:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='5', valid_fac='00040000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local2', location='/etc/syslog-ng/syslog-ng.conf:142:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:141:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local1', location='/etc/syslog-ng/syslog-ng.conf:141:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='5', valid_fac='00020000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local1', location='/etc/syslog-ng/syslog-ng.conf:141:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:144:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local4', location='/etc/syslog-ng/syslog-ng.conf:144:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='5', valid_fac='00100000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local4', location='/etc/syslog-ng/syslog-ng.conf:144:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:143:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local3', location='/etc/syslog-ng/syslog-ng.conf:143:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='5', valid_fac='00080000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local3', location='/etc/syslog-ng/syslog-ng.conf:143:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_loggen', location='/etc/syslog-ng/syslog-ng.conf:97:22', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: match() evaluation started; input='syslog-ng', pattern='.*[.]000', value='PROGRAM', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_loggen', location='/etc/syslog-ng/syslog-ng.conf:97:22', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Initial message parsing follows; Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='HOST', value='', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='MESSAGE', value='Error processing log message: <-1>Jun 9 11:23:47 testlogger-service-54956569bb-2c5wl testlogger: 2020-06-09 11:23:47,346 (main) INFO [c.b.s.r.s.TestLoggerServer] [trkId=] Test log message \x0a\x0a>@<\x0a<-1>Jun 9 11:23:47 testlogger-service-54956569bb-2c5wl testlogger: 2020-06-09 11:23:47,346 (main) INFO [c.b.s.r.s.TestLoggerServer] [trkId=] Test log message ', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='PROGRAM', value='syslog-ng', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='PID', value='28873', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='HOST_FROM', value='10.13.98.167', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='HOST', value='10.13.98.167/10.13.98.167', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Setting value; name='SOURCE', value='s_tcp_net_514', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:147:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local7', location='/etc/syslog-ng/syslog-ng.conf:147:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='5', valid_fac='00800000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local7', location='/etc/syslog-ng/syslog-ng.conf:147:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:146:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local6', location='/etc/syslog-ng/syslog-ng.conf:146:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='5', valid_fac='00400000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local6', location='/etc/syslog-ng/syslog-ng.conf:146:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:145:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local5', location='/etc/syslog-ng/syslog-ng.conf:145:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='5', valid_fac='00200000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local5', location='/etc/syslog-ng/syslog-ng.conf:145:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:142:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local2', location='/etc/syslog-ng/syslog-ng.conf:142:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='5', valid_fac='00040000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local2', location='/etc/syslog-ng/syslog-ng.conf:142:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:141:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local1', location='/etc/syslog-ng/syslog-ng.conf:141:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='5', valid_fac='00020000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local1', location='/etc/syslog-ng/syslog-ng.conf:141:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:144:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local4', location='/etc/syslog-ng/syslog-ng.conf:144:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='5', valid_fac='00100000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local4', location='/etc/syslog-ng/syslog-ng.conf:144:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: Requesting flow control; location='/etc/syslog-ng/syslog-ng.conf:143:32' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_local3', location='/etc/syslog-ng/syslog-ng.conf:143:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: facility() evaluation started; fac='5', valid_fac='00080000', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_local3', location='/etc/syslog-ng/syslog-ng.conf:143:32', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: >>>>>> filter rule evaluation begin; rule='f_loggen', location='/etc/syslog-ng/syslog-ng.conf:97:22', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: match() evaluation started; input='syslog-ng', pattern='.*[.]000', value='PROGRAM', msg='0x7f2af8003800' Jun 9 11:23:47 s_local@syslog syslog-ng[28873]: <<<<<< filter rule evaluation result; result='UNMATCHED - Dropping message from LogPipe', rule='f_loggen', location='/etc/syslog-ng/syslog-ng.conf:97:22', msg='0x7f2af8003800' Any help would be greatly appreciated. Thank you, Jason

3 9

Disk buffer getting wiped out
by Raghunath Adhyapak 15 Jun '20

15 Jun '20

Hi, We have syslog-ng forwarding logs to a remote system, with reliability enabled and disk buffer size set to 50GB in the configuration. However, the actual size of the partition is 30GB. We had overlooked the partition size that resulted in this error. When there is connection loss to remote device and partition gets filled to 100% due to syslog-ng disk buffer. syslog-ng continues to run, but even after we free some space in partition and destination is available, it doesn't push out logs to the destination. We have to restart syslog-ng service. After restarting, we observe that all buffer files are freed up immediately. These files were consuming about 20-30GB. Syslog-ng version using :3.24 Below is disk buffer configuration: mem-buf-size(10000) disk-buf-size(53687091200) reliable(yes) dir(/syslog/buffer) Questions: 1. Does syslog-ng allocate buffers on disk as much as required and in small chunks? Or will it try to allocate all of the specified buffer at one (in this case 50GB)? 2. Why couldn't syslog-ng resume operations after partition was freed up and destination was available? 3. Why did all buffer files get deleted, after a restart Thanks Raghu

5 10

Insider 2020-06: edge; log management layer; WSL;
by Peter Czanik (pczanik) 11 Jun '20

11 Jun '20

Dear syslog-ng users, This is the 82nd issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Syslog-ng on the edge --------------------- After many years of pushing all computing from on-site to the cloud or huge data centers, there is a new trend: edge computing. There can be many reasons, legal or practical, why data should be processed locally instead of being sent to a central location as soon as it is created. Edge computing was a central theme of the recently held Red Hat Summit. Luckily syslog-ng is well prepared for this use case right from the beginning. While most people only know that syslog-ng can act as a client or a server, it can also collect, process and forward log messages. In syslog-ng terminology it is called a relay, but on the edge you might want to combine server and a relay functionality into one. https://www.syslog-ng.com/community/b/blog/posts/syslog-ng-on-the-edge Creating a dedicated log management layer ----------------------------------------- Event logging is a central source of information both for IT security and operations, but different teams use different tools to collect and analyze log messages. The same log message is often collected by multiple applications. Having each team using different tools is complex, inefficient and makes systems less secure. Using a single application to create a dedicated log management layer independent of analytics instead, however, has multiple benefits. Using syslog-ng is a lot more flexible than most log aggregation tools provided by log analytics vendors. This is one of the reasons why my talks and blogs focused on how to make your life easier using its technical advantages. Of course, I am aware of the direct financial benefits as well. If you are interested in that part, talk to my colleagues on the business side. They can help you to calculate how much you can save on your SIEM licenses when syslog-ng collects log messages and ensures that only relevant messages reach your SIEM and only at a predicatively low message rate. You can learn more about this use case on our Optimizing SIEM page. In this blog, I will focus on a third aspect: simplifying complexity. This was the focus of many of my conference discussions before the COVID-19 pandemic. If we think a bit more about it, we can see that this is not really a third aspect, but a combination of the previous two instead. Using the flexibility of syslog-ng, we create a dedicated log management layer in front of different log analytics solutions. By reducing complexity, we can save in many ways: on computing and human resources, and on licensing when using commercial tools for log analysis as well. https://www.syslog-ng.com/community/b/blog/posts/creating-a-dedicated-log-m… I will cover this topic more in depth in my upcoming talk at the Pass the SALT conference: https://pass-the-salt.org/ Using syslog-ng in WSL ---------------------- Windows Subsystem for Linux (WSL) is an optional feature of Windows 10 for developers who want the power of Linux (especially the Linux shell) on their Windows desktops. Of course, it is more than just a shell: you can easily install and run any command line applications (but not GUI). As a Linux desktop user, I do not need WSL to access a Linux shell, but as I am often asked how syslog-ng runs in WSL, I finally gave it a try. The recurring questions are if syslog-ng runs at all in WSL and what the performance compared to syslog-ng installed on Linux is. As I run openSUSE Leap 15.1 as my main operating system on my laptop, I used that in WSL as well. I tested not just WSL 1, which has been generally available for years, but also the upcoming WSL 2, which brings tons of performance improvements. As WSL 2 involves virtualization, I also tested syslog-ng in Vmware Workstation running on Windows. In all cases, I used the latest syslog-ng 3.26 from my unofficial syslog-ng repository for openSUSE and a minimally modified syslog-ng.conf to enable the network source. Benchmarking was done both from localhost and from a small Xeon server on the local network, attached through Gigabit Ethernet. https://www.syslog-ng.com/community/b/blog/posts/using-syslog-ng-in-wsl CONFERENCES * ”Creating a dedicated log management layer” talk at the virtual Pass the SALT conference: https://pass-the-salt.org/ WEBINARS * Learn how to do custom HTTP REST API Authentication with syslog-ng: https://www.syslog-ng.com/event/customize-http-rest-apis-authentication-wit… * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Poll: Do you use any of the Java-based destination drivers of syslog-ng?
by Peter Czanik (pczanik) 10 Jun '20

10 Jun '20

Hi, Just posted a poll on Twitter, but answers are also very welcome here. Do you use any of the Java-based destination drivers of syslog-ng? HDFS, Kafka (not the C implementation) or Elasticsearch (not the elasticsearch-http() version). Or do you use the syslog-ng Java bindings with your own custom Java module? Let as know here or on Twitter at https://twitter.com/PCzanik/status/1270700394059104263 Also, please RT the above tweet to reach more syslog-ng users! Thanks, Peter Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0