- syslog-ng - lists.balabit.hu

flags(final) - Question
by Matthias Gruber 09 Feb '21

09 Feb '21

Hi! After searching why a log-message doesnt got the right destination path, I propably got a bit crossed up with the flag(final) statement, perhaps someone can enlight me :-) Following scenario (no valid syntax, only for description) filter(f_a) { match("something");}; destination(d_a) { file("/destination_a"); }: destination(d_b) { file("/destination_b"); }: destination(d_c) { file("/destination_c"); }: log { source(s_remote_default_syslog_udp); if filter(f_a) { destination(d_a); flag(final); } else { destination(d_b); flag(final); }; destination(d_c); }; What I expect is if filter->f_a hits the message is written in destination->d_a (what he do), then hits the final, and ends the log-statement What I discovered he also write destination->d_c Is that the expected behaviour? And if so, how to accomplish, that e.g. if the if or the else part hit, no destination -> d_c is written. Yeah I know, why the destination->d_c :-), correct I stumbled over this, normally after such a if/elif/else-Block I dunno have this destination->d_c line in my syslog-ng environments. Only a bit courious about it and seeking for an explanation. cheers Matthias ------------------------------------------------------------------------------------ METZLER Informationstechnologie Matthias Gruber IT-Infrastruktur & -Betrieb B. Metzler seel. Sohn & Co. Kommanditgesellschaft auf Aktien Untermainanlage 1 60329 Frankfurt am Main Telefon (0 69) 21 04 - 43 30 Telefax (0 69) 21 04 - 40 40 MGruber(a)metzler.com www.metzler.com Persönlich haftende Gesellschafter: Harald Illy, Friedrich von Metzler, Emmerich Müller, Gerhard Wiesheu Vorsitzender des Aufsichtsrats: Dr. Christoph Schücking Sitz der Gesellschaft: Frankfurt am Main, Handelsregister-Nr. HRB 27 515 Diese Nachricht ist vertraulich. Sollten Sie nicht der vorgesehene Empfänger sein, so bitten wir Sie höflich, dies unverzüglich dem Absender mitzuteilen und die Nachricht zu löschen. Es ist unzulässig, die Nachricht unbefugt weiterzuleiten oder zu kopieren. Da wir nicht die Echtheit oder Vollständigkeit der in dieser Nachricht enthaltenen Informationen garantieren oder zusichern können, sind die vorstehenden Ausführungen rechtlich nicht bindend. Eine Haftung hierfür wird ausgeschlossen. This message is confidential. If you are not the intended recipient, we kindly ask you to inform the sender and delete the information. Any unauthorised dissemination or copying hereof is prohibited. As we cannot guarantee or assure the genuineness or completeness of the information contained in this message, the statements set forth above are not legally binding. Accordingly we cannot accept any liability for their contents.

2 2

filtering based on text?
by Dan Egli 08 Feb '21

08 Feb '21

Hey folks, I've looked for a way to do this, and I guess my google-foo is weak today. I've got a situation where on one of my machines, an automated process FREQUENTLY calls sudo so it can gain the permissions it needs to do certain tasks. But that means my /var/log/messages is getting FILLED with sudo messages. I was hoping I could insert some kind of text filter that would allow me to shunt messages where one user (the automated process) calls sudo into another log file. It would basically need to be a nested filter, i.e.: If message_source = sudo then if user = X then log to auto_sudo.log If someone knows how this can be done, I'd appreciate it. Thanks! --- Dan

3 3

PostgreSQL destination - connection refused
by victorsugo 06 Feb '21

06 Feb '21

Hi, I'd like to store incoming log messages in a postgres db. Both db server and syslog-ng run in a docker container. I'm using docker-compose to manage containers. The syslog-ng is depending on the pg container, but... every time I start containers with docker-compose up -d, the syslog-ng starts, but can't connect to database. I found this in the /var/log/messages on the syslog-ng's container: err syslog-ng[1]: Error establishing SQL connection; type='pgsql', host='pgsrv.infra-net', port='', username='syslog', database='postgres', error='could not connect to server: Connection refused ... Is there any way to reconnect automatically or abort syslog-ng if it can't connect to the db? It is important if I don't want to lose some logs... (or do I need to forget the SQL backend if I want to operate a log server without loss of data??) Regards, Sent with [ProtonMail](https://protonmail.com) Secure Email.

1 1

problems with multiline log messages on ubuntu18
by Jerry Scharf 06 Feb '21

06 Feb '21

Hi, We have been struggling with problems with multiple log messages being broken apart. Our apps can send random things to the logger including entire stack traces. We have a classifier that sends unknown lines to their own log file, so this makes a complete mess. This was with the distro apt package. So I went to building 3.25.1 by hand. If I build it with nothing on the configure line, it processes the multiline log messages just fine but if I build it with the ubuntu18 debian rules, it breaks the same way. I narrowed it down to if --eneable-systemd is set it breaks. Here are some relevant things. DISTRIB_DESCRIPTION="Ubuntu 18.04.5 LTS" config cmd ../configure --prefix=/usr/local --sysconfdir=/etc/syslog-ng --enable-dynamic-linking --enable-json --enable-manpages --enable-python --enable-java=no --enable-mongodb=no --enable-legacy-mongodb-options=no SOURCE_REVISION=3.25.1-ir1 --enable-systemd=no attaching the conf and the file classifier (60...) thanks, jerry

2 1

Help with filtering at destination based on custom macro value match
by Akshay Joshi 30 Jan '21

30 Jan '21

My client is sending logs and it has the following config : *template t_global {template("<${PRI}>${LOGHOST} ${MSG_TAG}${MSGHDR}${MSG}\n"); };* *# Global logging remote destination:#-----------------------------------destination d_global_remote { tcp("proxy.dc.nuagedemo.net <http://proxy.dc.nuagedemo.net>" port(10514) template(t_global) tls(peer-verify(required-untrusted) ca-dir('/etc/default/bootstrap/keys') cert_file('/etc/default/bootstrap/keys/cert.pem') key-file('/etc/default/bootstrap/keys/key.pem') ) flags("threaded") );};source s_nuageDiag { file("/home/user/nuage/nuage_diagnostics_daemon.log" follow-freq(10) default-facility(local1) default-priority(info) tags("nuageDiag"));};rewrite w_nuageDiag { set("nuage-diag: ", value("MSG_TAG") condition(tags("nuageDiag"))); };* On the destination, I have this : *source s_network { tcp( port(10514) max-connections(1000) tls( peer-verify(required-untrusted) key-file("/opt/proxy/config/keys/proxy-Key.pem") cert_file("/opt/proxy/config/keys/proxyCert.pem") ca-dir("/opt/proxy/config/keys/proxy-CA.pem") ) );};* *filter nsg_diag { match("nuage-diag: " value("MSG_TAG"));};filter f_messages { (level(info..warn) and filter (nsg_diag)); };* *destination d_logs { file( "/var/log/syslog-ng/logs.txt" owner("root") group("root") perm(0777) );};log { source(s_sys); source(s_network); filter(f_messages); destination(d_logs); };* I can write logs locally without the filtering. But with filtering, it does not match "nuage-diag: " macro. This "MSG_TAG" does not seem to be a standard header but a custom one. I couldn't find many straightforward examples on forums etc.. as well. I am missing a trick or two config-wise for sure. Any pointers / help will be much appreciated. -- Regards, Akshay Joshi

3 2

syslog-ng won't talk to systemd?
by Dan Egli 26 Jan '21

26 Jan '21

I'm trying to setup syslog-ng on a systemd system. But even though I have several things writing to syslog, syslog-ng doesn't write ANYTHING to the /var/log/messages except kernel messages and it's own info. So I searched around, and found the systemd-journal() source. So I tried it. And I get an error that says syslog-ng can't initialize the logger plugin: # syslog-ng -F [2021-01-23T23:25:36.525097] The configuration must not contain more than one systemd-journal() source; [2021-01-23T23:25:36.525157] Error initializing journal_reader; [2021-01-23T23:25:36.525173] Error initializing message pipeline; plugin_name='systemd-journal', location='/etc/syslog-ng/syslog-ng.conf:26:5' Here's the config, basically copied from the syslog-ng.com page: @version: 3.26 # # Syslog-ng default configuration file for Gentoo Linux # https://bugs.gentoo.org/426814 @include "scl.conf" options { threaded(yes); chain_hostnames(no); stats_freq(43200); mark_freq(3600); }; source s_journald { systemd-journal(prefix(".SDATA.journald.")); }; source src { system(); internal(); file("/proc/kmsg"); }; destination messages { file("/var/log/messages"); }; destination console_all { file("/dev/tty12"); }; log { source(src); destination(messages); }; log { source(s_journald); destination(messages); }; log { source(src); destination(console_all); }; What am I doing wrong? I am POSITIVE things are sending to the syslog sub-system. I have ensured several programs have sent something. But without the systemd-journal line syslog-ng loads fine, recording /proc/kmsg and syslog-ng messages but nothing else. I am completely stumped here.

2 3

Polling additional machines
by Dan Egli 23 Jan '21

23 Jan '21

I have an interesting setup that was proposed to me, and while I _THINK_ syslog-ng can handle it I don't know for sure, let alone HOW. There are multiple service-specific machines in the network, behind the firewall. I'll just use the services they provide as their hostnames in this example. So firewall fw is a small linux machine running syslog-ng and fail2ban. Now fail2ban works off of the log files, but it also wants to apply changes to the firewall rules locally. So either www has its firewall rules while smtp has its rules and imap has a third set (ugly) or fail2ban can run on fw and modify the rules there. Trick is, in order to do that, all the logs have to be accessible in real time so that fail2ban can see them. My thought was to have syslog-ng on each sub machine somehow report to the syslog-ng on fw. Then fw's syslog-ng can write the files that fail2ban wants to read from. What would be the best way to go about this? I am still learning syslog-ng so if this is something simple, I'm sorry.

2 1

syslog-ng on CentOS 7 Aarch64
by Peter Czanik (pczanik) 20 Jan '21

20 Jan '21

Hi, I just learned that RHEL 7 on Aarch64 (64bit Arm) is end of life. Copr, where I build syslog-ng packages, removes RHEL/CentOS 7 Aarch64 packages in about half a year and it is not possible to build new packages there any more. CentOS 7 on the other hand keeps going on Aarch64 (using CentOS 8 kernel and CentOS 7 userland). There might be a workaround for the situation (using a different build service), but it's quite a lot extra effort. So, before investing any time and effort into keep building CentOS 7 Aarch64, I want to know, if there is a significant number of syslog-ng users, who keep using CentOS 7 on Aarch64. You can respond here or to my Twitter poll at: https://twitter.com/PCzanik/status/1351816930345291776 Thanks, Peter Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Signed Syslog Messages RFC 5848
by Alexandre Santos 20 Jan '21

20 Jan '21

Hi, Are there any plans to add Signed Syslog Messages RFC 5848 to syslog-ng? Best Regards, Alex

3 4

Signed Syslog Messages RFC 5848
by Secure-Logging, Generic 19 Jan '21

19 Jan '21

Hi Alexandre, as a replacement for signed messages you might have a look at the secure logging module of syslog-ng. This module authentically encrypts each message, so you get an integrity guarantee which is stronger than a pure signature. You can then forward the message to a backend for verification and further processing. Regards, Stephan The information in this e-mail is confidential. The contents may not be disclosed or used by anyone other than the addressee. Access to this e-mail by anyone else is unauthorised. If you are not the intended recipient, please notify Airbus immediately and delete this e-mail. Airbus cannot accept any responsibility for the accuracy or completeness of this e-mail as it has been sent over public networks. If you have any concerns over the content of this message or its Accuracy or Integrity, please contact Airbus immediately. All outgoing e-mails from Airbus are checked using regularly updated virus scanning software but you should take whatever measures you deem to be appropriate to ensure that this message and any attachments are virus free.

1 0