March 2020 - syslog-ng - lists.balabit.hu

MARK messages and filtering
by Alexandre Santos 19 Mar '20

19 Mar '20

Hi, I have the following part of syslog configuration: destination d_localfile_kernel { file("/var/log/kernel.log" flags(syslog-protocol) ); }; filter f_localfile_kernel_kern { facility(kern) and level(info .. emerg); }; filter f_localfile_kernel { filter(f_localfile_kernel_kern); }; log { source(s_src); filter(f_localfile_kernel); destination(d_localfile_kernel); flags(flow-control); }; And I see in my kernel.log file: <46>1 2020-03-18T18:35:28+00:00 localhost - - - [meta sequenceId="1"] -- MARK -- <46>1 2020-03-18T18:55:28+00:00 localhost - - - [meta sequenceId="2"] -- MARK -- <46>1 2020-03-18T19:15:28+00:00 localhost - - - [meta sequenceId="3"] -- MARK -- <46>1 2020-03-18T19:35:28+00:00 localhost - - - [meta sequenceId="4"] -- MARK -- <46>1 2020-03-18T19:55:28+00:00 localhost - - - [meta sequenceId="5"] -- MARK -- <46>1 2020-03-18T20:15:28+00:00 localhost - - - [meta sequenceId="6"] -- MARK -- <46>1 2020-03-18T20:35:28+00:00 localhost - - - [meta sequenceId="7"] -- MARK -- <46>1 2020-03-18T20:55:28+00:00 localhost - - - [meta sequenceId="8"] -- MARK -- <46>1 2020-03-18T21:15:28+00:00 localhost - - - [meta sequenceId="9"] -- MARK -- I was not expecting to see syslog facility messages coming out, since I am filtering by kernel facility. Is this expected behavior? Thanks & regards, Alex

2 1

Insider 2020-03: syslog-ng-stable; macOS; BastilleBSD;
by Peter Czanik (pczanik) 19 Mar '20

19 Mar '20

Dear syslog-ng users, This is the 79th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Introducing the syslog-ng-stable RPM repositories ------------------------------------------------- For many years – especially after syslog-ng changed to a rolling release model – users I talked to asked for up-to-date RPM packages. They also asked for a separate repository for each new release to avoid surprises (a new release might accidentally or even intentionally break old features) and to be able to use a given release if they want to (“if it works, do not fix it”). That is how my unofficial RPM repositories were born. Recently some long-time syslog-ng users and members of the Splunk community started to ask for a repository, which always has the latest syslog-ng version available. Most users still prefer to use separate repositories. That is how I came up with the idea for the syslog-ng-stable repository: I push a new release to this new rolling repo only after at least a week of delay. This is enough to spot most major problems. Once the delay is over and everything seems to be OK, I can push the latest release to the syslog-ng-stable repo. If there is a bigger problem, I can skip the release in the stable repo or wait for a fix. https://www.syslog-ng.com/community/b/blog/posts/introducing-the-syslog-ng-… How to install syslog-ng on macOS --------------------------------- A member of the syslog-ng community wrote a guide on how to enable syslog-ng on macOS: “My goal was to send any kind of Mac logs to a big data platform (Devo). Mac already comes with syslogd, which is the Apple System Log server. It is basically a daemon that processes syslog messages but to be honest, it’s pretty old and basic. As a result, I decided to try syslog-ng.” https://medium.com/macoclock/how-to-install-syslog-ng-on-macos-965127f1c05c Running syslog-ng in BastilleBSD -------------------------------- Bastille is a container management system for FreeBSD. If you are coming from a Linux world, it is a bit like Docker or Podman / Buildah from Red Hat, at least some of its functionality. I learned about BastilleBSD right before my Christmas holidays. Currently my primary work platform is Linux and I am just preparing to learn about Kubernetes and OpenShift. I planned not to do anything work-related during my holidays – which is quite difficult, if your hobby heavily overlaps with your work. Having some strong FreeBSD roots (started to use FreeBSD in 1994), BastilleBSD arrived just on time to be a good excuse to do something IT-related :-) https://www.syslog-ng.com/community/b/blog/posts/running-syslog-ng-in-basti… NEW WHITE PAPERS * Upgrade to State-of-the-Art Log Management: https://www.syslog-ng.com/whitepaper/log-management-essentials8137754/ WEBINARS * Learn how to easily integrate Azure Sentinel into your log-data strategy: https://www.brighttalk.com/webcast/16207/390780 * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

3.26 question
by Peter Czanik (pczanik) 18 Mar '20

18 Mar '20

Hi, Do you use syslog-ng 3.26? Before I add a new release to my syslog-ng-stable repo normally I check for feedback on the release. But here I have a problem. There are no bug reports about syslog-ng 3.26, which is good. On the other hand I could not find any reports that someone is already using 3.26. Do you have any feedback on 3.26? Either positive or negative? Thanks, Peter Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

use /dev/log on systemd machines
by Matus UHLAR - fantomas 18 Mar '20

18 Mar '20

Hello, if I understand properly, the system() source opens /dev/log socket when machine does not run systemd, am I right? Can I force syslog-ng to open /dev/log socket in this case or have I to configure it explicitly? -- Matus UHLAR - fantomas, uhlar(a)fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. I intend to live forever - so far so good.

2 1

cmake vs autotools
by Chris Packham 18 Mar '20

18 Mar '20

Hi, Question from the buildroot mailing list [1]. Is syslog-ng transitioning to cmake or will both cmake an autotools be supported for the foreseeable future. It's not an issue either way buildroot supports building with either cmake or autotools. But if autotools will be retired we may as well switch now to avoid hassles with future updates. Thanks, Chris -- [1] - http://lists.busybox.net/pipermail/buildroot/2020-March/277459.html

3 2

syslog-ng 3.26 RPM packages
by Peter Czanik (pczanik) 10 Mar '20

10 Mar '20

Hi, A bit slower than usual, but my unofficial syslog-ng 2.26 RPM packages are now available. I'm still traveling, so I did not have a chance for much testing. Any feedback is very welcome! * openSUSE/SLES: https://build.opensuse.org/project/show/home:czanik:syslog-ng326 * fedora/RHEL/CentOS: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng326/ Peter Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Fwd: facility override
by Alexandre Santos 09 Mar '20

09 Mar '20

Hi, I tried to rewrite PRI but the macro is not writable: Mar 09 11:27:31 localhost syslog-ng[31986]: Error parsing rewrite expression, PRI is read-only, it cannot be changed in rewrite rules in /etc/syslog-ng/syslog-ng.conf: Mar 09 11:27:31 localhost syslog-ng[31986]: 359 }; Mar 09 11:27:31 localhost syslog-ng[31986]: 360 # end of localFile: netconf-commands Mar 09 11:27:31 localhost syslog-ng[31986]: 361 Mar 09 11:27:31 localhost syslog-ng[31986]: 362 Mar 09 11:27:31 localhost syslog-ng[31986]: 363 # remote: test Mar 09 11:27:31 localhost syslog-ng[31986]: 364---> rewrite r_pri{set("189", value("PRI"));}; Mar 09 11:27:31 localhost syslog-ng[31986]: 364---> ^ Mar 09 11:27:31 localhost syslog-ng[31986]: 365 destination d_test_udp { Mar 09 11:27:31 localhost syslog-ng[31986]: 366 syslog("192.168.122.1" transport("udp") ip-protocol(4) port(514) Mar 09 11:27:31 localhost syslog-ng[31986]: 367 suppress(5) Mar 09 11:27:31 localhost syslog-ng[31986]: 368 disk-buffer( Mar 09 11:27:31 localhost syslog-ng[31986]: 369 mem-buf-size(2097152) I tried to use the following configuration to make a facility override local0 -> local7: # remote: test template t_test { template("$(if ('${LEVEL_NUM}' == '0') '<184>1' $(if ('${LEVEL_NUM}' == '1') '<185>1' $(if ('${LEVEL_NUM}' == '2') '<186>1' $(if ('${LEVEL_NUM}' == '3') '<187>1' $(if ('${LEVEL_NUM}' == '4') '<188>1' $(if ('${LEVEL_NUM}' == '5') '<189>1' $(if ('${LEVEL_NUM}' == '6') '<190>1' '<191>1' ))))))) ${ISODATE} ${HOST} ${PROGRAM} ${PID} $(if ('${MSGID}' == '') '-' '${MSGID}') $(if ('${SDATA}' == '') '-' '${SDATA}') ${MESSAGE}"); }; destination d_test_udp { network("192.168.122.1" transport("udp") ip-protocol(4) port(514) template ("t_test") ); }; filter f_remote_test_udp_local0 { facility(local0) and level(debug .. emerg); }; log { source(s_src); filter(f_remote_test_udp_local0); destination(d_test_udp); flags(flow-control); }; # end of remote server: test The problem is that I get a double header in the syslog server: Mar 9 11:41:52 127.0.0.1 root[31353] <190>1 2020-03-09T11:41:52+00:00 127.0.0.1 root 31353 - - This is a Test 6 Mar 9 11:41:53 127.0.0.1 root[31404] <190>1 2020-03-09T11:41:53+00:00 127.0.0.1 root 31404 - - This is a Test 7 Mar 9 11:41:53 127.0.0.1 root[31448] <190>1 2020-03-09T11:41:53+00:00 127.0.0.1 root 31448 - - This is a Test 8 Mar 9 11:41:54 127.0.0.1 root[31449] <190>1 2020-03-09T11:41:54+00:00 127.0.0.1 root 31449 - - This is a Test 9 Mar 9 11:41:54 127.0.0.1 root[31468] <190>1 2020-03-09T11:41:54+00:00 127.0.0.1 root 31468 - - This is a Test 10 Mar 9 11:41:55 127.0.0.1 root[31485] <190>1 2020-03-09T11:41:55+00:00 127.0.0.1 root 31485 - - This is a Test 11 Mar 9 11:41:55 127.0.0.1 root[31526] <190>1 2020-03-09T11:41:55+00:00 127.0.0.1 root 31526 - - This is a Test 12 Mar 9 11:41:55 127.0.0.1 root[31567] <190>1 2020-03-09T11:41:55+00:00 127.0.0.1 root 31567 - - This is a Test 13 Can you help me? Thanks and regards, Alex On Mon, Jan 20, 2020 at 07:49 AM Attila Szakacs (aszakacs) < Attila.Szakacs(a)oneidentity.com> wrote: > Hi Alex, > > I think ${PRI} does exactly that. > > PRI > Description: The priority and facility encoded as a 2 or 3 digit decimal number as it is present in syslog messages. > > Regards, > Attila > ________________________________ > From: syslog-ng <syslog-ng-bounces at lists.balabit.hu <https://lists.balabit.hu/mailman/listinfo/syslog-ng>> on behalf of Alexandre Santos <alexandre.rosas.santos at gmail.com <https://lists.balabit.hu/mailman/listinfo/syslog-ng>> > Sent: Friday, January 17, 2020 7:56 PM > To: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu <https://lists.balabit.hu/mailman/listinfo/syslog-ng>> > Subject: [syslog-ng] facility override > > CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. > > Hi, > > Is there a way of overriding the facility of the messages on the destination, besides using a template like: > { template("$(if ('${LEVEL_NUM}' == '2') '<186>1' $(if ('${LEVEL_NUM}' == '3') '<187>1' $(if ('${LEVEL_NUM}' == '4') '<188>1' ${HOST}\\@`HOSTIPV4` ${PROGRAM} ${PID} ${SDATA} ${MESSAGE}\n"); }; > > Thanks in advance, > Alex >

2 2

restart vs reload
by Alexandre Santos 09 Mar '20

09 Mar '20

Hi, I am trying to dynamically change the output message format between RFC3164 and RFC5424. root@localhost:~# diff /etc/syslog-ng/syslog-ng.conf /root/syslog-ng.conf.bak 365c365 < network("192.168.122.1" transport("udp") ip-protocol(4) port(514) --- > syslog("192.168.122.1" transport("udp") ip-protocol(4) port(514) If I use: "/usr/sbin/service syslog-ng restart" it works fine, but if I use "/usr/sbin/syslog-ng-ctl reload", the message format does not change. I am using syslog-ng version 3.19 Any thoughts? Thanks & Regards, Alex

2 1

my unofficial 3.26 RPM packages delayed
by Peter Czanik (pczanik) 05 Mar '20

05 Mar '20

Hi, Just a heads up. I'm traveling and at my current location wifi is too slow and unstable to do package building. Once I have a stable wifi I'll first build them in my syslog-ng-githead repositories and once everything builds (I have to disable Python support on SLES 12 as Python 3.4 is too old) I'll create the usual versioned repositories. So lets hope, that wifi will work better at my next hotel / conference and then I can prepare my usual openSUSE/SLES & Fedora/RHEL/CentOS packages. Peter Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Not able to link syslog-ng libraries through gcc
by Nitish Saboo 05 Mar '20

05 Mar '20

Hi, I compiled syslog-ng-3.25.1 using the following code: tar -xvzf syslog-ng_3.25.1.tar.gz mkdir -p syslog-ng_3.25.1/build cd syslog-ng_3.25.1/build ../configure --prefix=/home/nsaboo/Desktop/Cproject/syslog/syslog-ng_3.25.1/install make make install Post installation,the folder structure looks like this: nsaboo@ubuntu:~/Desktop/Cproject/syslog/syslog-ng-3.25.1/install$ ls -lrt total 28 drwxrwxr-x 2 nsaboo nsaboo 4096 Feb 11 04:08 sbin drwxrwxr-x 2 nsaboo nsaboo 4096 Feb 11 04:08 bin drwxr-xr-x 2 nsaboo nsaboo 4096 Feb 11 04:08 var drwxr-xr-x 3 nsaboo nsaboo 4096 Feb 11 04:08 share drwxrwxr-x 4 nsaboo nsaboo 4096 Feb 11 04:08 include drwxr-xr-x 3 nsaboo nsaboo 4096 Feb 11 04:08 etc drwxrwxr-x 4 nsaboo nsaboo 4096 Feb 18 00:35 lib nsaboo@ubuntu:~/Desktop/Cproject/syslog/syslog-ng-3.25.1/install$ I have a C code base (syslog-node.c) and header file (node.h) that is internally calling syslog-ng patterndb apis': I have a main.c file that calls methods in 'syslog-node.c'. My header file has following methods: typedef void (*key_value_cb)(const char* key, const char* value, size_t value_len, int data); int initialize_engine(const char* filename, const char* module_path, key_value_cb cb); int reload_pattern_db(const char* filename, key_value_cb cb); int load_pattern_db(const char* filename, key_value_cb cb); void match(const char* pattern, size_t pattern_len, const char* program, size_t program_len); int shutdown(); I am calling following gcc command to create a binary but it is erroring out: gcc -L/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/lib lsyslog-ng -L/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/lib/syslog-ng ldbparser -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/include/syslog-ng -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/include -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/ -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/build/ -I/usr/local/include/eventlog/ syslog-node.c main.c -o main Error ===== nsaboo@ubuntu:~/Desktop/Cproject$ gcc -L/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/lib -lsyslog-ng -L/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/lib/syslog-ng -ldbparser -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/include/syslog-ng -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/include -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/ -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/build/ -I/usr/local/include/eventlog/ syslog-node.c main.c -o main /tmp/ccam1ao5.o: In function `pdbtool_pdb_emit_accumulate': syslog-node.c:(.text+0x55): undefined reference to `logmsg_registry' syslog-node.c:(.text+0x73): undefined reference to `nv_table_foreach' /tmp/ccam1ao5.o: In function `match': syslog-node.c:(.text+0x93): undefined reference to `log_msg_new_empty' syslog-node.c:(.text+0xb0): undefined reference to `log_msg_set_value' syslog-node.c:(.text+0xc9): undefined reference to `log_msg_set_value' syslog-node.c:(.text+0xdf): undefined reference to `pattern_db_process' syslog-node.c:(.text+0xeb): undefined reference to `log_msg_unref' /tmp/ccam1ao5.o: In function `reload_pattern_db': syslog-node.c:(.text+0x1e2): undefined reference to `pattern_db_free' syslog-node.c:(.text+0x1e7): undefined reference to `pattern_db_new' syslog-node.c:(.text+0x20b): undefined reference to `pattern_db_reload_ruleset' syslog-node.c:(.text+0x225): undefined reference to `pattern_db_set_emit_func' /tmp/ccam1ao5.o: In function `load_pattern_db': syslog-node.c:(.text+0x257): undefined reference to `pattern_db_free' syslog-node.c:(.text+0x25c): undefined reference to `pattern_db_new' syslog-node.c:(.text+0x280): undefined reference to `pattern_db_reload_ruleset' syslog-node.c:(.text+0x29a): undefined reference to `pattern_db_set_emit_func' /tmp/ccam1ao5.o: In function `initialize_engine': syslog-node.c:(.text+0x2ba): undefined reference to `app_startup' syslog-node.c:(.text+0x2c4): undefined reference to `msg_init' syslog-node.c:(.text+0x2cb): undefined reference to `resolvedConfigurablePaths' syslog-node.c:(.text+0x2d0): undefined reference to `resolved_configurable_paths_init' syslog-node.c:(.text+0x2db): undefined reference to `resolvedConfigurablePaths' syslog-node.c:(.text+0x2e5): undefined reference to `cfg_new' syslog-node.c:(.text+0x2fb): undefined reference to `cfg_set_global_paths' syslog-node.c:(.text+0x311): undefined reference to `cfg_load_module' syslog-node.c:(.text+0x327): undefined reference to `cfg_load_module' syslog-node.c:(.text+0x32c): undefined reference to `pattern_db_global_init' syslog-node.c:(.text+0x347): undefined reference to `pattern_db_free' syslog-node.c:(.text+0x34c): undefined reference to `pattern_db_new' syslog-node.c:(.text+0x370): undefined reference to `pattern_db_reload_ruleset' syslog-node.c:(.text+0x38a): undefined reference to `pattern_db_set_emit_func' /tmp/ccam1ao5.o: In function `shutdown': syslog-node.c:(.text+0x39a): undefined reference to `app_shutdown' collect2: error: ld returned 1 exit status Those functions are available in the shared libraries.nm command output shows the same nsaboo@ubuntu:~/Desktop/Cproject$ nm syslog/syslog-ng-3.25.1/install/lib/syslog-ng/libdbparser.so | grep T | grep 'pattern_db_set_emit_func' 0000000000010d40 T pattern_db_set_emit_func nsaboo@ubuntu:~/Desktop/Cproject$ nm syslog/syslog-ng-3.25.1/install/lib/syslog-ng/libdbparser.so | grep T | grep 'pattern_db_new' 0000000000010ef0 T pattern_db_new nsaboo@ubuntu:~/Desktop/Cproject$ nm syslog/syslog-ng-3.25.1/install/lib/syslog-ng/libdbparser.so | grep T | grep 'pattern_db_reload_ruleset' 0000000000010cc0 T pattern_db_reload_ruleset nsaboo@ubuntu:~/Desktop/Cproject$ nm syslog/syslog-ng-3.25.1/install/lib/libsyslog-ng.so.0 | grep T | grep 'log_msg_set_value' 0000000000080df0 T log_msg_set_value 00000000000811d0 T __log_msg_set_value_by_name 0000000000081220 T log_msg_set_value_indirect nsaboo@ubuntu:~/Desktop/Cproject$ nm syslog/syslog-ng-3.25.1/install/lib/libsyslog-ng.so | grep T | grep 'msg_init' 0000000000051990 T msg_init nsaboo@ubuntu:~/Desktop/Cproject$ nm syslog/syslog-ng-3.25.1/install/lib/libsyslog-ng.so | grep T | grep 'cfg_new' 0000000000039200 T cfg_new 0000000000039360 T cfg_new_snippet 0000000000039390 T cfg_new_subordinate I understand this is the C linking error where I am including all the dynamic shared libraries and header files, but not able to figure out what is causing this issue. Can someone please help me to determine what am I missing here ? Thanks, Nitish

1 1