- syslog-ng - lists.balabit.hu

fedora/rhel & logrotate
by Peter Czanik (pczanik) 18 Feb '20

18 Feb '20

Hi, Recently I received a bug report about logrotate and RHEL 8: https://bugzilla.redhat.com/show_bug.cgi?id=1802165 In RHEL 8 /etc/logroate.d/syslog is now rsyslog specific and thus conflicts with the same file coming from syslog-ng. I changed my test repo https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng-githead/ in a way that RHEL 7 keeps using /etc/logroate/syslog, but all the Fedora and RHEL 8 packages change the logroate file name to /etc/logroate/syslog-ng. Of course this means, that there is no conflict with rsyslog and there are two logrotate scripts for the same set of logs. I am trying to test it on my own as well on RHEL 8, but any additional testing and feedback are very welcome! Thanks, Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 1

syslog-ng doesn't log if system date is older (in the past) than date on last shutdown
by Abhi Arora 18 Feb '20

18 Feb '20

Hi, I have one embedded device running Linux which doesn't have RTC. So it loses date and time after every halt. *uname -a:* *Linux f1 4.9.175+g0746276 #1 SMP PREEMPT Tue Jan 14 03:09:19 UTC 2020 armv7l armv7l armv7l GNU/Linux* I have observed after every reboot, my board starts with some older time and my application running in user-space corrects the system time from a timestamp saved on a file (which is updated every hour). Application also connects with some server to get latest time. I have noticed that if system date and time is less than date and time which it had at last shutdown, then syslog doesn't get any logs from my applications till it has a time atleast greater than date and time which it had at last shutdown. My source was "systemd_journal()" and I changed it to "unix-dgram("/dev/log");". However, after changing it to unix-dgram, I don't see any logs even if my system date and time are correct. *"ls -l /dev/log":> lrwxrwxrwx 1 root root 28 Sep 30 13:28 /dev/log -> /run/systemd/journal/dev-log * *This is my old /etc/syslog-ng/scl.conf* : *@include 'scl/*/*.conf'@define java-module-dir "`module-install-dir`/java-modules"* ******************** This by* *cat /etc/syslog-ng/syslog-ng.conf* *@version: 3.15## Syslog-ng configuration file, compatible with default Debian syslogd# installation. Originally written by anonymous (I can't find his name)# Revised, and rewrited by me (SZALAY Attila <sasa(a)debian.org <sasa(a)debian.org>>)# First, set some global options.options { chain_hostnames(off); flush_lines(0); use_dns(no); use_fqdn(no); owner("root"); group("adm"); perm(0640); stats_freq(0); bad_hostname("^gconfd$"); keep_hostname(yes); keep_timestamp(yes);};######################### Sources######################### This is the default behavior of sysklogd package# Logs may come from unix stream, but not from another machine.#source s_src { systemd_journal(); internal(); file("/proc/kmsg" program_override("kernel"));};# If you wish to get logs from remote machine you should uncomment# this and comment the above source line.##source s_net { tcp(ip(127.0.0.1) port(1000) authentication(required) encrypt(allow)); };######################### Destinations######################### First some standard logfile#destination d_auth { file("/var/log/auth.log"); };destination d_cron { file("/var/log/cron.log"); };destination d_daemon { file("/var/log/daemon.log"); };destination d_kern { file("/var/log/kern.log"); };destination d_lpr { file("/var/log/lpr.log"); };destination d_mail { file("/var/log/mail.log"); };destination d_syslog { file("/var/log/syslog"); };destination d_user { file("/var/log/user.log"); };destination d_uucp { file("/var/log/uucp.log"); };# This files are the log come from the mail subsystem.#destination d_mailinfo { file("/var/log/mail/mail.info <http://mail.info/>"); };destination d_mailwarn { file("/var/log/mail/mail.warn"); };destination d_mailerr { file("/var/log/mail/mail.err"); };# Logging for INN news system#destination d_newscrit { file("/var/log/news/news.crit"); };destination d_newserr { file("/var/log/news/news.err"); };destination d_newsnotice { file("/var/log/news/news.notice"); };# Some 'catch-all' logfiles.#destination d_debug { file("/var/log/debug"); };destination d_error { file("/var/log/error"); };destination d_messages { file("/var/log/messages"); };# The root's console.#destination d_console { usertty("root"); };# Virtual console.#destination d_console_all { file("/dev/tty10"); };# The named pipe /dev/xconsole is for the nsole' utility. To use it,# you must invoke nsole' with the -file' option:## $ xconsole -file /dev/xconsole [...]#destination d_xconsole { pipe("/dev/xconsole"); };# Send the messages to an other host##destination d_net { tcp("127.0.0.1" port(1000) authentication(on) encrypt(on) log_fifo_size(1000)); };# Debian onlydestination d_ppp { file("/var/log/ppp.log"); };######################### Filters######################### Here's come the filter options. With this rules, we can set which# message go where.filter f_dbg { level(debug); };filter f_info { level(info); };filter f_notice { level(notice); };filter f_warn { level(warn); };filter f_err { level(err); };filter f_crit { level(crit .. emerg); };filter f_alllevel { level(debug, info, notice, warn,err,crit,emerg) and not filter(f_daemon);};filter f_debug { level(debug) and not facility(auth, authpriv, news, mail); };filter f_error { level(err .. emerg) ; };filter f_messages { level(info,notice,warn) and not facility(auth,authpriv,cron,daemon,mail,news); };filter f_auth { facility(auth, authpriv) and not filter(f_debug); };filter f_cron { facility(cron) and not filter(f_debug); };filter f_daemon { facility(daemon) and not filter(f_debug); };filter f_kern { facility(kern) and not filter(f_debug); };filter f_lpr { facility(lpr) and not filter(f_debug); };filter f_local { facility(local0, local1, local3, local4, local5, local6, local7) and not filter(f_debug); };filter f_mail { facility(mail) and not filter(f_debug); };filter f_news { facility(news) and not filter(f_debug); };filter f_syslog3 { not facility(auth, authpriv, mail) and not filter(f_debug); };filter f_user { facility(user) and not filter(f_debug); };filter f_uucp { facility(uucp) and not filter(f_debug); };filter f_cnews { level(notice, err, crit) and facility(news); };filter f_cother { level(debug, info, notice, warn) or facility(daemon, mail); };filter f_ppp { facility(local2) and not filter(f_debug); };filter f_console { level(warn .. emerg); };######################### Log paths########################log { source(s_src); filter(f_auth); destination(d_auth); };log { source(s_src); filter(f_cron); destination(d_cron); };log { source(s_src); filter(f_daemon); destination(d_daemon); };log { source(s_src); filter(f_kern); destination(d_Gateway); };log { source(s_src); filter(f_lpr); destination(d_lpr); };log { source(s_src); filter(f_syslog3); destination(d_syslog); };log { source(s_src); filter(f_user); destination(d_user); };log { source(s_src); filter(f_uucp); destination(d_uucp); };log { source(s_src); filter(f_mail); destination(d_mail); };#log { source(s_src); filter(f_mail); filter(f_info); destination(d_mailinfo); };#log { source(s_src); filter(f_mail); filter(f_warn); destination(d_mailwarn); };#log { source(s_src); filter(f_mail); filter(f_err); destination(d_mailerr); };log { source(s_src); filter(f_news); filter(f_crit); destination(d_newscrit); };log { source(s_src); filter(f_news); filter(f_err); destination(d_newserr); };log { source(s_src); filter(f_news); filter(f_notice); destination(d_newsnotice); };#log { source(s_src); filter(f_cnews); destination(d_console_all); };#log { source(s_src); filter(f_cother); destination(d_console_all); };#log { source(s_src); filter(f_ppp); destination(d_ppp); };log { source(s_src); filter(f_debug); destination(d_debug); };log { source(s_src); filter(f_error); destination(d_error); };log { source(s_src); filter(f_messages); destination(d_messages); };log { source(s_src); filter(f_console); destination(d_console_all); destination(d_xconsole); };log { source(s_src); filter(f_crit); destination(d_console); };# All messages send to a remote site##log { source(s_src); destination(d_net); };# GentApp/GatewayAppdestination d_Gateway { file("/var/log/Gateway.log"template("$FULLDATE $FULLHOST $PROGRAM $PID [$FACILITY.$LEVEL] $MESSAGE\n") ); };filter f_Gateway { match("Gate*") or match("Gent*") or match("Notifier*") or match("Alarm*"); };* *log { source(s_src); filter(f_Gateway);filter(f_alllevel); destination(d_Gateway);} ;* Please help me in debugging the issue and fixing it.

2 2

Set log directory based on the source
by Frank DiGennaro 18 Feb '20

18 Feb '20

Hello; I'm somewhat of a newbie when it comes to syslog-ng so forgive me if the question is somewhat basic. I am running syslog-ng 3.25 on both server and clients and would like to set a variable based on the source. This is what I have: @define server-logdir "/var/log" @define client-logdir "/var/log/CLIENTS/$YEAR/$MONTH_ABBREV/$DAY/$HOST" @define logdir "" source s_local { logdir `server-logdir`; *No! ... ... }; source s_network { logdir `client-logdir`; *No! ... ... }; destination d_local0 { file( "`logdir`/local/local0.log" ); }; The goal is to set 'logdir' to either 'server-logdir' or 'client-logdir' and use it in the destination. I know there are probably several ways that this can be accomplished but I think this was will cut down on the number of lines in the configuration. So my question Is this. How do I set 'logdir' to either 'server-logdir' or 'client-logdir' in the source definition? Thanks; DiGennaro Frank DiGennaro 301-676-8193 (C) <mailto:frank@digennaro.com> frank(a)digennaro.com

3 5

[Question]: Initialisation step in syslog-ng-3.25.1
by Nitish Saboo 18 Feb '20

18 Feb '20

I am using syslog-ng-3.25.1.I am initialising syslog-ng in the following manner: int initialize_engine(const gchar* _module_path) { resolvedConfigurablePaths.initial_module_path = _module_path; app_startup(); msg_init(TRUE); [*]//resolved_configurable_paths_init(&resolvedConfigurablePaths); configuration = cfg_new(0x0302); cfg_set_global_paths(configuration); cfg_load_module(configuration, "basicfuncs"); cfg_load_module(configuration, "syslogformat"); pattern_db_global_init(); return 0; } I have assigned value to 'resolvedConfigurablePaths.initial_module_path' to provide module-path location. Do I have to add this step [*]'resolved_configurable_paths_init(&resolvedConfigurablePaths)' that is commented currently in initialize_engine method ? Thanks, Nitish

2 1

Warnings and error while loading default.xml in syslog-ng-3.25.1
by Nitish Saboo 17 Feb '20

17 Feb '20

Hi, I am using syslog-ng version 3.25.1.Getting following Warnings and error while initialising syslog-ng engine: [2020-02-13T10:47:29.627899] WARNING: due to a bug in versions before syslog-ng 3.8numeric comparison operators like '!=' in filter expressions were evaluated as string operators. This is fixed in syslog-ng 3.8. As we are operating in compatibility mode, syslog-ng will exhibit the buggy behaviour as previous versions until you bump the @version value in your configuration file; [2020-02-13T10:47:29.627968] WARNING: due to a bug in versions before syslog-ng 3.8numeric comparison operators like '!=' in filter expressions were evaluated as string operators. This is fixed in syslog-ng 3.8. As we are operating in compatibility mode, syslog-ng will exhibit the buggy behaviour as previous versions until you bump the @version value in your configuration file; [2020-02-13T10:47:29.628059] WARNING: due to a bug in versions before syslog-ng 3.8numeric comparison operators like '!=' in filter expressions were evaluated as string operators. This is fixed in syslog-ng 3.8. As we are operating in compatibility mode, syslog-ng will exhibit the buggy behaviour as previous versions until you bump the @version value in your configuration file; [2020-02-13T10:47:29.631090] Error parsing pattern database file; filename='/opt/tap-parsing/patterns/default.xml', error='/opt/tap-parsing/patterns/default.xml:17274:22: Joining rulesets with mismatching program name sets, program=proxysg' 1)For the following warnings, to which version I have to bump up the configuration file ? 2020-02-13T10:47:29.627899] WARNING: due to a bug in versions before syslog-ng 3.8numeric comparison operators like '!=' in filter expressions were evaluated as string operators. This is fixed in syslog-ng 3.8. As we are operating in compatibility mode, syslog-ng will exhibit the buggy behaviour as previous versions until you bump the @version value in your configuration file; [2020-02-13T10:47:29.627968] WARNING: due to a bug in versions before syslog-ng 3.8numeric comparison operators like '!=' in filter expressions were evaluated as string operators. This is fixed in syslog-ng 3.8. As we are operating in compatibility mode, syslog-ng will exhibit the buggy behaviour as previous versions until you bump the @version value in your configuration file; [2020-02-13T10:47:29.628059] WARNING: due to a bug in versions before syslog-ng 3.8numeric comparison operators like '!=' in filter expressions were evaluated as string operators. This is fixed in syslog-ng 3.8. As we are operating in compatibility mode, syslog-ng will exhibit the buggy behaviour as previous versions until you bump the @version value in your configuration file; Currrently the configuration version is the following: configuration = cfg_new(0x0302) Do I have to change it to '0x0319' as defined in 'lib/versioning.h' ? 2)The same default.xml file was getting loaded correctly in syslog-ng-3.6.2 and syslog-ng-3.7.1 but getting following error while loading same default.xml in syslog-ng3.25.1 2020-02-13T10:47:29.631090] Error parsing pattern database file; filename='/home/nsaboo/abc/default.xml', error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with mismatching program name sets, program=proxysg'. What can be the reason for this error ? Thanks, Nitish

3 5

Insider 2020-02: Portability; secure logging; Mac support; RPM;
by Peter Czanik (pczanik) 13 Feb '20

13 Feb '20

Dear syslog-ng users, This is the 78th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Keeping syslog-ng portable -------------------------- I define syslog-ng as an “Enhanced logging daemon with a focus on portability and high-performance central log collection”. One of the original goals of syslog-ng was portability: running the same application on a wide variety of architectures and operating systems. After one of my talks mentioning syslog-ng, I was asked how we ensure that syslog-ng stays portable when all the CI infrastructure focus on 64bit x86 architecture and Linux. You can learn my answer from: https://www.syslog-ng.com/community/b/blog/posts/keeping-syslog-ng-portable Why choose syslog-ng over rsyslog -------------------------------- A question I often receive is ‘what are the differences between rsyslog and syslog-ng?’ It’s a little tricky to answer. First, because my experience is mostly with syslog-ng, and because there are many similarities between the two projects. This is where the syslog-ng users help me. They can clearly explain from firsthand experience why they chose syslog-ng. The following blog post includes some of the most common reasons why they choose syslog-ng. https://www.syslog-ng.com/community/b/blog/posts/why-chose-syslog-ng-over-r… Secure logging with syslog-ng ----------------------------- In his presentation at FOSDEM, Stephan Marwedel mentioned a new syslog-ng feature he is working on, namely “(...) the design, implementation, and configuration of the secure logging service. Its aim is to provide tamper evident logging, i.e., to adequately protect log records of an information system against tampering and to provide a sensor indicating attack attempts.” (Marwedel, S. Secure logging with syslog-ng. (n.d.). Retrieved from https://fosdem.org/2020/schedule/event/security_secure_logging_with_syslog_… more about the presentation, and access the slides at https://fosdem.org/2020/schedule/event/security_secure_logging_with_syslog_… A pull request implementing this feature was already sent to syslog-ng on GitHub. You can follow its progress at https://github.com/syslog-ng/syslog-ng/pull/3121 State of syslog-ng on Mac ------------------------- Mac support is a returning question among syslog-ng users, especially when I talk to users in the US. For recent releases, each commit is automatically tested on macOS. However, there is not much information available on Mac support. Recently, I bought a MacBook to be able to test and document syslog-ng on Mac. Here are my first experiences and some future plans. https://www.syslog-ng.com/community/b/blog/posts/state-of-syslog-ng-on-mac Installing the latest syslog-ng on openSUSE, RHEL and other RPM distributions -------------------------------------- The syslog-ng application is included in all major Linux distributions, and you can usually install syslog-ng from the official repositories. If the core functionality of syslog-ng meets your needs, use the package in your distribution repository (yum install syslog-ng), and you can stop reading here. However, if you want to use the features of newer syslog-ng versions, for example, sending log messages to Elasticsearch or Apache Kafka, you have to either compile syslog-ng from source, or install it from unofficial repositories. This post explains how to do that. https://www.syslog-ng.com/community/b/blog/posts/installing-latest-syslog-n… NEW WHITE PAPERS * Upgrade to State-of-the-Art Log Management: https://www.syslog-ng.com/whitepaper/log-management-essentials8137754/ WEBINARS * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Getting error : .../libsyslog-ng.so: file not recognized: File truncated
by Nitish Saboo 11 Feb '20

11 Feb '20

Hi, I compiled the syslog-ng-3.7.1.tar.gz on a Ubuntu 16.04 machine .The syslog-ng got compiled successfully on this machine. ---------------------------------------------------------------------- Libraries have been installed in: /home/anassa/nitish/syslog-ng-3.7.1/install/lib If you ever happen to want to link against installed libraries in a given directory, LIBDIR, you must either use libtool, and specify the full pathname of the library, or use the `-LLIBDIR' flag during linking and do at least one of the following: - add LIBDIR to the `LD_LIBRARY_PATH' environment variable during execution - add LIBDIR to the `LD_RUN_PATH' environment variable during linking - use the `-Wl,-rpath -Wl,LIBDIR' linker flag - have your system administrator add LIBDIR to `/etc/ld.so.conf' See any operating system documentation about shared libraries for more information, such as the ld(1) and ld.so(8) manual pages. ---------------------------------------------------------------------- --------------------------------------------------------------------Libraries have been installed in: /home/anassa/nitish/syslog-ng-3.7.1/install/lib/syslog-ng If you ever happen to want to link against installed libraries in a given directory, LIBDIR, you must either use libtool, and specify the full pathname of the library, or use the `-LLIBDIR' flag during linking and do at least one of the following: - add LIBDIR to the `LD_LIBRARY_PATH' environment variable during execution - add LIBDIR to the `LD_RUN_PATH' environment variable during linking - use the `-Wl,-rpath -Wl,LIBDIR' linker flag - have your system administrator add LIBDIR to `/etc/ld.so.conf' See any operating system documentation about shared libraries for more information, such as the ld(1) and ld.so(8) manual pages. ---------------------------------------------------------------------- -Ubuntu:~/nitish/syslog-ng-3.7.1/install/lib$ gcc --version gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609 Copyright (C) 2015 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. My project got compiled in 'syslog-ng-3.7.1' folder on machine Ubuntu 16.04. I have a different machine with Ubuntu 14.04.I copied 'syslog-ng-3.7.1' folder on this machine (Ubuntu 14.04). In this machine I tried compiling my CGO project (Go and C) where my Go code is calling the C wrapper code that internally call syslog-ng api's. I have defined all the CFLAGS and LDFLAGS in my Go code for the run-timne dependencies. At the time of compilation it is failing with the following error: Error ===== lib/../deps/syslog/syslog-ng-3.7.1/install/lib/libsyslog-ng.so: file not recognized: File truncated [11:36:44][shellscript] collect2: error: ld returned 1 exit status [11:36:44][shellscript] make: *** [main] Error 2 The file 'libsyslog-ng.so' is present at the location 'lib/../deps/syslog/syslog-ng-3.7.1/install/lib/': total 3152 drwxrwxr-x 2 anassa anassa 4096 Feb 5 16:47 pkgconfig -rwxr-xr-x 1 anassa anassa 3208176 Feb 5 16:47 libsyslog-ng-3.7.so.0.0.0 lrwxrwxrwx 1 anassa anassa 25 Feb 5 16:47 libsyslog-ng.so -> libsyslog-ng-3.7.so.0.0.0 -rwxr-xr-x 1 anassa anassa 1155 Feb 5 16:47 libsyslog-ng.la lrwxrwxrwx 1 anassa anassa 25 Feb 5 16:47 libsyslog-ng-3.7.so.0 -> libsyslog-ng-3.7.so.0.0.0 drwxrwxr-x 3 anassa anassa 4096 Feb 5 16:47 python2.7 drwxrwxr-x 3 anassa anassa 4096 Feb 5 16:47 syslog-ng Can someone please guide me what could be the possible reasons to get this error ? What am I missing here ? Thanks, Nitish

2 5

syslog doesn't log if system date is older than date on last reboot
by Abhi Arora 07 Feb '20

07 Feb '20

Hi, I have one embedded device running Linux which doesn't have RTC. So it loses date and time after every reboot. uname -a returns: Linux f1 4.9.175+g0746276 #1 SMP PREEMPT Tue Jan 14 03:09:19 UTC 2020 armv7l armv7l armv7l GNU/Linux I have observed after every reboot, my board starts with some older time and my application running in user-space corrects the system time from a timestamp saved on a file (which is updated every hour). Application also connects with some server to get latest time. I have noticed that if system date and time is less than date and time which it had at last shutdown, then syslog doesn't get any logs from my applications till it has a time atleast greater than date and time which it had at last shutdown. Please help me in debugging the issue and fixing it.

4 16

Problem creating boot.log file
by Frank DiGennaro 04 Feb '20

04 Feb '20

All; I am running syslog-ng 3.25 and am trying to log the boot messages to boot.log. This is my configuration: @define logdir "/var/log/CLIENTS/$YEAR/$MONTH_ABBREV/$DAY/$HOST" filter f_boot { facility( local7 ) }; destination d_boot { file( "`logdir`/boot.log" ); }; log { source( s_local ); source( s_network ); filter( f_boot ); destination( d_boot ); }; source s_local { system( ); internal( ); }; The problem that I have is that the file is being created in /var/log, not in "logdir". Perhaps the reason for that could be that the syslog-ng service is started later in the boot process. Any insight at all would be greatly appreciated. Thanks; Frank

1 0

C_* macros with UTC time?
by Matus UHLAR - fantomas 04 Feb '20

04 Feb '20

Hello, I have syslog-ng OSE 3.8 on debian 9 server (soon to be upgraded). when I receive syslog lines using the syslog protocol, I found out that C_* macros seem to contain date in UTC format (one hour than our current zone). I have tried to use following template: "${C_DATE} ${R_DATE} HOST ${HOST} FULLHOST ${FULLHOST} HOST_FROM ${HOST_FROM} FULLHOST_FROM ${FULLHOST_FROM} ${MESSAGE}\n" with following result: Jan 31 10:42:06 Jan 31 11:42:07 HOST fantomas FULLHOST fantomas HOST_FROM 195.80.174.185 FULLHOST_FROM 195.80.174.185 Configuration reload request received, reloading configuration; Is there any reason why C_DATE should be in different time zone than R_DATE? timezones are not configured on either of servers, both servers are in CET (DST+1) thank you -- Matus UHLAR - fantomas, uhlar(a)fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Fighting for peace is like fucking for virginity...

2 4