- syslog-ng - lists.balabit.hu

Warnings and error while loading default.xml in syslog-ng-3.25.1
by Nitish Saboo 17 Feb '20

17 Feb '20

Hi, I am using syslog-ng version 3.25.1.Getting following Warnings and error while initialising syslog-ng engine: [2020-02-13T10:47:29.627899] WARNING: due to a bug in versions before syslog-ng 3.8numeric comparison operators like '!=' in filter expressions were evaluated as string operators. This is fixed in syslog-ng 3.8. As we are operating in compatibility mode, syslog-ng will exhibit the buggy behaviour as previous versions until you bump the @version value in your configuration file; [2020-02-13T10:47:29.627968] WARNING: due to a bug in versions before syslog-ng 3.8numeric comparison operators like '!=' in filter expressions were evaluated as string operators. This is fixed in syslog-ng 3.8. As we are operating in compatibility mode, syslog-ng will exhibit the buggy behaviour as previous versions until you bump the @version value in your configuration file; [2020-02-13T10:47:29.628059] WARNING: due to a bug in versions before syslog-ng 3.8numeric comparison operators like '!=' in filter expressions were evaluated as string operators. This is fixed in syslog-ng 3.8. As we are operating in compatibility mode, syslog-ng will exhibit the buggy behaviour as previous versions until you bump the @version value in your configuration file; [2020-02-13T10:47:29.631090] Error parsing pattern database file; filename='/opt/tap-parsing/patterns/default.xml', error='/opt/tap-parsing/patterns/default.xml:17274:22: Joining rulesets with mismatching program name sets, program=proxysg' 1)For the following warnings, to which version I have to bump up the configuration file ? 2020-02-13T10:47:29.627899] WARNING: due to a bug in versions before syslog-ng 3.8numeric comparison operators like '!=' in filter expressions were evaluated as string operators. This is fixed in syslog-ng 3.8. As we are operating in compatibility mode, syslog-ng will exhibit the buggy behaviour as previous versions until you bump the @version value in your configuration file; [2020-02-13T10:47:29.627968] WARNING: due to a bug in versions before syslog-ng 3.8numeric comparison operators like '!=' in filter expressions were evaluated as string operators. This is fixed in syslog-ng 3.8. As we are operating in compatibility mode, syslog-ng will exhibit the buggy behaviour as previous versions until you bump the @version value in your configuration file; [2020-02-13T10:47:29.628059] WARNING: due to a bug in versions before syslog-ng 3.8numeric comparison operators like '!=' in filter expressions were evaluated as string operators. This is fixed in syslog-ng 3.8. As we are operating in compatibility mode, syslog-ng will exhibit the buggy behaviour as previous versions until you bump the @version value in your configuration file; Currrently the configuration version is the following: configuration = cfg_new(0x0302) Do I have to change it to '0x0319' as defined in 'lib/versioning.h' ? 2)The same default.xml file was getting loaded correctly in syslog-ng-3.6.2 and syslog-ng-3.7.1 but getting following error while loading same default.xml in syslog-ng3.25.1 2020-02-13T10:47:29.631090] Error parsing pattern database file; filename='/home/nsaboo/abc/default.xml', error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with mismatching program name sets, program=proxysg'. What can be the reason for this error ? Thanks, Nitish

3 5

Insider 2020-02: Portability; secure logging; Mac support; RPM;
by Peter Czanik (pczanik) 13 Feb '20

13 Feb '20

Dear syslog-ng users, This is the 78th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Keeping syslog-ng portable -------------------------- I define syslog-ng as an “Enhanced logging daemon with a focus on portability and high-performance central log collection”. One of the original goals of syslog-ng was portability: running the same application on a wide variety of architectures and operating systems. After one of my talks mentioning syslog-ng, I was asked how we ensure that syslog-ng stays portable when all the CI infrastructure focus on 64bit x86 architecture and Linux. You can learn my answer from: https://www.syslog-ng.com/community/b/blog/posts/keeping-syslog-ng-portable Why choose syslog-ng over rsyslog -------------------------------- A question I often receive is ‘what are the differences between rsyslog and syslog-ng?’ It’s a little tricky to answer. First, because my experience is mostly with syslog-ng, and because there are many similarities between the two projects. This is where the syslog-ng users help me. They can clearly explain from firsthand experience why they chose syslog-ng. The following blog post includes some of the most common reasons why they choose syslog-ng. https://www.syslog-ng.com/community/b/blog/posts/why-chose-syslog-ng-over-r… Secure logging with syslog-ng ----------------------------- In his presentation at FOSDEM, Stephan Marwedel mentioned a new syslog-ng feature he is working on, namely “(...) the design, implementation, and configuration of the secure logging service. Its aim is to provide tamper evident logging, i.e., to adequately protect log records of an information system against tampering and to provide a sensor indicating attack attempts.” (Marwedel, S. Secure logging with syslog-ng. (n.d.). Retrieved from https://fosdem.org/2020/schedule/event/security_secure_logging_with_syslog_… more about the presentation, and access the slides at https://fosdem.org/2020/schedule/event/security_secure_logging_with_syslog_… A pull request implementing this feature was already sent to syslog-ng on GitHub. You can follow its progress at https://github.com/syslog-ng/syslog-ng/pull/3121 State of syslog-ng on Mac ------------------------- Mac support is a returning question among syslog-ng users, especially when I talk to users in the US. For recent releases, each commit is automatically tested on macOS. However, there is not much information available on Mac support. Recently, I bought a MacBook to be able to test and document syslog-ng on Mac. Here are my first experiences and some future plans. https://www.syslog-ng.com/community/b/blog/posts/state-of-syslog-ng-on-mac Installing the latest syslog-ng on openSUSE, RHEL and other RPM distributions -------------------------------------- The syslog-ng application is included in all major Linux distributions, and you can usually install syslog-ng from the official repositories. If the core functionality of syslog-ng meets your needs, use the package in your distribution repository (yum install syslog-ng), and you can stop reading here. However, if you want to use the features of newer syslog-ng versions, for example, sending log messages to Elasticsearch or Apache Kafka, you have to either compile syslog-ng from source, or install it from unofficial repositories. This post explains how to do that. https://www.syslog-ng.com/community/b/blog/posts/installing-latest-syslog-n… NEW WHITE PAPERS * Upgrade to State-of-the-Art Log Management: https://www.syslog-ng.com/whitepaper/log-management-essentials8137754/ WEBINARS * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Getting error : .../libsyslog-ng.so: file not recognized: File truncated
by Nitish Saboo 11 Feb '20

11 Feb '20

Hi, I compiled the syslog-ng-3.7.1.tar.gz on a Ubuntu 16.04 machine .The syslog-ng got compiled successfully on this machine. ---------------------------------------------------------------------- Libraries have been installed in: /home/anassa/nitish/syslog-ng-3.7.1/install/lib If you ever happen to want to link against installed libraries in a given directory, LIBDIR, you must either use libtool, and specify the full pathname of the library, or use the `-LLIBDIR' flag during linking and do at least one of the following: - add LIBDIR to the `LD_LIBRARY_PATH' environment variable during execution - add LIBDIR to the `LD_RUN_PATH' environment variable during linking - use the `-Wl,-rpath -Wl,LIBDIR' linker flag - have your system administrator add LIBDIR to `/etc/ld.so.conf' See any operating system documentation about shared libraries for more information, such as the ld(1) and ld.so(8) manual pages. ---------------------------------------------------------------------- --------------------------------------------------------------------Libraries have been installed in: /home/anassa/nitish/syslog-ng-3.7.1/install/lib/syslog-ng If you ever happen to want to link against installed libraries in a given directory, LIBDIR, you must either use libtool, and specify the full pathname of the library, or use the `-LLIBDIR' flag during linking and do at least one of the following: - add LIBDIR to the `LD_LIBRARY_PATH' environment variable during execution - add LIBDIR to the `LD_RUN_PATH' environment variable during linking - use the `-Wl,-rpath -Wl,LIBDIR' linker flag - have your system administrator add LIBDIR to `/etc/ld.so.conf' See any operating system documentation about shared libraries for more information, such as the ld(1) and ld.so(8) manual pages. ---------------------------------------------------------------------- -Ubuntu:~/nitish/syslog-ng-3.7.1/install/lib$ gcc --version gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609 Copyright (C) 2015 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. My project got compiled in 'syslog-ng-3.7.1' folder on machine Ubuntu 16.04. I have a different machine with Ubuntu 14.04.I copied 'syslog-ng-3.7.1' folder on this machine (Ubuntu 14.04). In this machine I tried compiling my CGO project (Go and C) where my Go code is calling the C wrapper code that internally call syslog-ng api's. I have defined all the CFLAGS and LDFLAGS in my Go code for the run-timne dependencies. At the time of compilation it is failing with the following error: Error ===== lib/../deps/syslog/syslog-ng-3.7.1/install/lib/libsyslog-ng.so: file not recognized: File truncated [11:36:44][shellscript] collect2: error: ld returned 1 exit status [11:36:44][shellscript] make: *** [main] Error 2 The file 'libsyslog-ng.so' is present at the location 'lib/../deps/syslog/syslog-ng-3.7.1/install/lib/': total 3152 drwxrwxr-x 2 anassa anassa 4096 Feb 5 16:47 pkgconfig -rwxr-xr-x 1 anassa anassa 3208176 Feb 5 16:47 libsyslog-ng-3.7.so.0.0.0 lrwxrwxrwx 1 anassa anassa 25 Feb 5 16:47 libsyslog-ng.so -> libsyslog-ng-3.7.so.0.0.0 -rwxr-xr-x 1 anassa anassa 1155 Feb 5 16:47 libsyslog-ng.la lrwxrwxrwx 1 anassa anassa 25 Feb 5 16:47 libsyslog-ng-3.7.so.0 -> libsyslog-ng-3.7.so.0.0.0 drwxrwxr-x 3 anassa anassa 4096 Feb 5 16:47 python2.7 drwxrwxr-x 3 anassa anassa 4096 Feb 5 16:47 syslog-ng Can someone please guide me what could be the possible reasons to get this error ? What am I missing here ? Thanks, Nitish

2 5

syslog doesn't log if system date is older than date on last reboot
by Abhi Arora 07 Feb '20

07 Feb '20

Hi, I have one embedded device running Linux which doesn't have RTC. So it loses date and time after every reboot. uname -a returns: Linux f1 4.9.175+g0746276 #1 SMP PREEMPT Tue Jan 14 03:09:19 UTC 2020 armv7l armv7l armv7l GNU/Linux I have observed after every reboot, my board starts with some older time and my application running in user-space corrects the system time from a timestamp saved on a file (which is updated every hour). Application also connects with some server to get latest time. I have noticed that if system date and time is less than date and time which it had at last shutdown, then syslog doesn't get any logs from my applications till it has a time atleast greater than date and time which it had at last shutdown. Please help me in debugging the issue and fixing it.

4 16

Problem creating boot.log file
by Frank DiGennaro 04 Feb '20

04 Feb '20

All; I am running syslog-ng 3.25 and am trying to log the boot messages to boot.log. This is my configuration: @define logdir "/var/log/CLIENTS/$YEAR/$MONTH_ABBREV/$DAY/$HOST" filter f_boot { facility( local7 ) }; destination d_boot { file( "`logdir`/boot.log" ); }; log { source( s_local ); source( s_network ); filter( f_boot ); destination( d_boot ); }; source s_local { system( ); internal( ); }; The problem that I have is that the file is being created in /var/log, not in "logdir". Perhaps the reason for that could be that the syslog-ng service is started later in the boot process. Any insight at all would be greatly appreciated. Thanks; Frank

1 0

C_* macros with UTC time?
by Matus UHLAR - fantomas 04 Feb '20

04 Feb '20

Hello, I have syslog-ng OSE 3.8 on debian 9 server (soon to be upgraded). when I receive syslog lines using the syslog protocol, I found out that C_* macros seem to contain date in UTC format (one hour than our current zone). I have tried to use following template: "${C_DATE} ${R_DATE} HOST ${HOST} FULLHOST ${FULLHOST} HOST_FROM ${HOST_FROM} FULLHOST_FROM ${FULLHOST_FROM} ${MESSAGE}\n" with following result: Jan 31 10:42:06 Jan 31 11:42:07 HOST fantomas FULLHOST fantomas HOST_FROM 195.80.174.185 FULLHOST_FROM 195.80.174.185 Configuration reload request received, reloading configuration; Is there any reason why C_DATE should be in different time zone than R_DATE? timezones are not configured on either of servers, both servers are in CET (DST+1) thank you -- Matus UHLAR - fantomas, uhlar(a)fantomas.sk ; http://www.fantomas.sk/ Warning: I wish NOT to receive e-mail advertising to this address. Varovanie: na tuto adresu chcem NEDOSTAVAT akukolvek reklamnu postu. Fighting for peace is like fucking for virginity...

2 4

unsubscribe
by Zhang, Husen 03 Feb '20

03 Feb '20

-----Original Message----- From: syslog-ng <syslog-ng-bounces(a)lists.balabit.hu> On Behalf Of Peter Kokai (pkokai) Sent: Monday, February 3, 2020 7:33 AM To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: EXTERNAL: Re: [syslog-ng] Not finding header files in syslog-ng-3.25.1 as compared to syslog-ng-3.6.2 Hello, There is no dev doc about it, as you might noticed. Probably no replace that with this answer exists. Luckily git remembers: git log --follow -p -- <path> git log --follow -p -- modules/dbparser/patterndb-int.h commit 1931dabf07b79a9b10d5902b5ad0b7f0509e7594 Author: Balazs Scheidler <bazsi77(a)gmail.com> Date: Wed Jul 29 15:10:06 2015 +0200 dbparser: get rid off patterndb-int.h As we have the broken-down header files, this is not needed anymore. Signed-off-by: Balazs Scheidler <balazs.scheidler(a)balabit.com> This commit might gave you a hint, which headers it was replaced with. Alternative idea: simply check which header provides the function you need. git log --follow -p -- src/tags.h commit ee1d0d7f0720c3ded0d2bfaf22ca6e42d669b7a6 Author: Balazs Scheidler <bazsi77(a)gmail.com> Date: Thu Sep 9 13:48:42 2010 +0200 separate directory hierarchy into lib/, modules/ and syslog-ng/ It was more and more cumbersome to compile syslog-ng during development, since a core change always recompiled (or just relinked) the modules as well even when that was not strictly necessary. (make install in src/) This patch reorganizes the directory hierarchy, lib is where the core lives modules where plugins are and syslog-ng is where the main program resides. The change here more simple, just moving the actual file. (Note tags.h was moved twice since your version, and now it is under lib/logmsg/) -- Kokan On Mon, Feb 03, 2020 at 05:34:21PM +0530, Nitish Saboo wrote: > CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe. > > Hi, > > I was using syslog version: syslog-ng-3.6.2. > In that particular version of my c wrapper code I was including the following header files: > > #include <modules/dbparser/patterndb-int.h> > #include <syslog-ng/tags.h> > > When I upgraded the syslog-ng version to 3.25.1,I don't see these > header file in the modules and include folder respectively post > installation > > Can someone please help me clarify, what these files have been replaced with? > > Thanks, > Nitish > ______________________________________________________________________ > ________ Member info: > https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flist > s.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7Cpeter > .kokai%40oneidentity.com%7C0616c4f0e4cc4cd1d26f08d7a8a13d03%7C91c369b5 > 1c9e439c989c1867ec606603%7C0%7C0%7C637163282794906164&sdata=mn27bw > mBoTLzPsp%2FeL648GZ81vo%2FfZMrfAeNdfwX5bk%3D&reserved=0 > Documentation: > https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.b > alabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data > =02%7C01%7Cpeter.kokai%40oneidentity.com%7C0616c4f0e4cc4cd1d26f08d7a8a > 13d03%7C91c369b51c9e439c989c1867ec606603%7C0%7C0%7C637163282794916099& > amp;sdata=W5ip5K3TXAoux%2BBRNPoJpw19tQoxaSHF0nkLMc37%2BFQ%3D&reser > ved=0 > FAQ: > https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.b > alabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7Cpeter.kokai%40one > identity.com%7C0616c4f0e4cc4cd1d26f08d7a8a13d03%7C91c369b51c9e439c989c > 1867ec606603%7C0%7C0%7C637163282794916099&sdata=UZs%2FG%2Fg%2BoUES > RozPNJOfyUwDKYpB5gzQ9jKc4XysbSY%3D&reserved=0 > ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

1 0

Not finding header files in syslog-ng-3.25.1 as compared to syslog-ng-3.6.2
by Nitish Saboo 03 Feb '20

03 Feb '20

Hi, I was using syslog version: syslog-ng-3.6.2. In that particular version of my c wrapper code I was including the following header files: #include <modules/dbparser/patterndb-int.h> #include <syslog-ng/tags.h> When I upgraded the syslog-ng version to 3.25.1,I don't see these header file in the modules and include folder respectively post installation Can someone please help me clarify, what these files have been replaced with? Thanks, Nitish

3 2

Memory not freeing up in syslog-ng-3.6.2 during reload
by Nitish Saboo 29 Jan '20

29 Jan '20

Hi, I am using syslog-ng version 3.6.2: syslog-ng --version syslog-ng 3.6.2 Installer-Version: 3.6.2 Revision: Compile-Date: May 2 2019 01:20:32 Available-Modules: afmongodb,afstomp,afuser,csvparser,afprog,afamqp,cryptofuncs,afsocket,graphite,dbparser,affile,syslogformat,afsocket-notls,system-source,confgen,pseudofile,linux-kmsg-format,basicfuncs Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off I initialised syslog-ng with the following configuration: int initialize_engine(const gchar* filename, const gchar* _module_path) { module_path = _module_path; app_startup(); msg_init(TRUE); configuration = cfg_new(0x0302); plugin_load_module("basicfuncs", configuration, NULL); plugin_load_module("syslogformat", configuration, NULL); pattern_db_global_init(); return 0; } int reload_pattern_db(const gchar* filename, key_value_cb cb) { if(patterndb!= NULL){ pattern_db_free(patterndb); } patterndb = pattern_db_new(); pattern_db_reload_ruleset(patterndb, configuration, filename); pattern_db_set_emit_func(patterndb, pdbtool_pdb_emit_accumulate, cb); return 0; } When I reload the syslog-ng, the memory keeps on increasing.Looks like syslog-ng is not freeing up the memory.It continues to occupy the memory unless the process is restarted. Just want to understand if it is a sign of memory leak or default behavior of syslog-ng to not free-up memory during reload ? Thanks, Nitish

2 15

which platform do you use for syslog-ng?
by Peter Czanik (pczanik) 28 Jan '20

28 Jan '20

Hi, I am often asked on which platforms people are running syslog-ng. Based on mailing list questions, github issues and so on I have some guesses, but every once in a while I run a Twitter poll: https://twitter.com/PCzanik/status/1222089870782484480 Please answer it and share it, so more people can answer it! Thanks, Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0