- syslog-ng - lists.balabit.hu

Not able to link syslog-ng libraries through gcc
by Nitish Saboo 05 Mar '20

05 Mar '20

Hi, I compiled syslog-ng-3.25.1 using the following code: tar -xvzf syslog-ng_3.25.1.tar.gz mkdir -p syslog-ng_3.25.1/build cd syslog-ng_3.25.1/build ../configure --prefix=/home/nsaboo/Desktop/Cproject/syslog/syslog-ng_3.25.1/install make make install Post installation,the folder structure looks like this: nsaboo@ubuntu:~/Desktop/Cproject/syslog/syslog-ng-3.25.1/install$ ls -lrt total 28 drwxrwxr-x 2 nsaboo nsaboo 4096 Feb 11 04:08 sbin drwxrwxr-x 2 nsaboo nsaboo 4096 Feb 11 04:08 bin drwxr-xr-x 2 nsaboo nsaboo 4096 Feb 11 04:08 var drwxr-xr-x 3 nsaboo nsaboo 4096 Feb 11 04:08 share drwxrwxr-x 4 nsaboo nsaboo 4096 Feb 11 04:08 include drwxr-xr-x 3 nsaboo nsaboo 4096 Feb 11 04:08 etc drwxrwxr-x 4 nsaboo nsaboo 4096 Feb 18 00:35 lib nsaboo@ubuntu:~/Desktop/Cproject/syslog/syslog-ng-3.25.1/install$ I have a C code base (syslog-node.c) and header file (node.h) that is internally calling syslog-ng patterndb apis': I have a main.c file that calls methods in 'syslog-node.c'. My header file has following methods: typedef void (*key_value_cb)(const char* key, const char* value, size_t value_len, int data); int initialize_engine(const char* filename, const char* module_path, key_value_cb cb); int reload_pattern_db(const char* filename, key_value_cb cb); int load_pattern_db(const char* filename, key_value_cb cb); void match(const char* pattern, size_t pattern_len, const char* program, size_t program_len); int shutdown(); I am calling following gcc command to create a binary but it is erroring out: gcc -L/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/lib lsyslog-ng -L/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/lib/syslog-ng ldbparser -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/include/syslog-ng -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/include -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/ -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/build/ -I/usr/local/include/eventlog/ syslog-node.c main.c -o main Error ===== nsaboo@ubuntu:~/Desktop/Cproject$ gcc -L/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/lib -lsyslog-ng -L/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/lib/syslog-ng -ldbparser -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/include/syslog-ng -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/install/include -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/ -I/home/nsaboo/Desktop/Cproject/syslog/syslog-ng-3.25.1/build/ -I/usr/local/include/eventlog/ syslog-node.c main.c -o main /tmp/ccam1ao5.o: In function `pdbtool_pdb_emit_accumulate': syslog-node.c:(.text+0x55): undefined reference to `logmsg_registry' syslog-node.c:(.text+0x73): undefined reference to `nv_table_foreach' /tmp/ccam1ao5.o: In function `match': syslog-node.c:(.text+0x93): undefined reference to `log_msg_new_empty' syslog-node.c:(.text+0xb0): undefined reference to `log_msg_set_value' syslog-node.c:(.text+0xc9): undefined reference to `log_msg_set_value' syslog-node.c:(.text+0xdf): undefined reference to `pattern_db_process' syslog-node.c:(.text+0xeb): undefined reference to `log_msg_unref' /tmp/ccam1ao5.o: In function `reload_pattern_db': syslog-node.c:(.text+0x1e2): undefined reference to `pattern_db_free' syslog-node.c:(.text+0x1e7): undefined reference to `pattern_db_new' syslog-node.c:(.text+0x20b): undefined reference to `pattern_db_reload_ruleset' syslog-node.c:(.text+0x225): undefined reference to `pattern_db_set_emit_func' /tmp/ccam1ao5.o: In function `load_pattern_db': syslog-node.c:(.text+0x257): undefined reference to `pattern_db_free' syslog-node.c:(.text+0x25c): undefined reference to `pattern_db_new' syslog-node.c:(.text+0x280): undefined reference to `pattern_db_reload_ruleset' syslog-node.c:(.text+0x29a): undefined reference to `pattern_db_set_emit_func' /tmp/ccam1ao5.o: In function `initialize_engine': syslog-node.c:(.text+0x2ba): undefined reference to `app_startup' syslog-node.c:(.text+0x2c4): undefined reference to `msg_init' syslog-node.c:(.text+0x2cb): undefined reference to `resolvedConfigurablePaths' syslog-node.c:(.text+0x2d0): undefined reference to `resolved_configurable_paths_init' syslog-node.c:(.text+0x2db): undefined reference to `resolvedConfigurablePaths' syslog-node.c:(.text+0x2e5): undefined reference to `cfg_new' syslog-node.c:(.text+0x2fb): undefined reference to `cfg_set_global_paths' syslog-node.c:(.text+0x311): undefined reference to `cfg_load_module' syslog-node.c:(.text+0x327): undefined reference to `cfg_load_module' syslog-node.c:(.text+0x32c): undefined reference to `pattern_db_global_init' syslog-node.c:(.text+0x347): undefined reference to `pattern_db_free' syslog-node.c:(.text+0x34c): undefined reference to `pattern_db_new' syslog-node.c:(.text+0x370): undefined reference to `pattern_db_reload_ruleset' syslog-node.c:(.text+0x38a): undefined reference to `pattern_db_set_emit_func' /tmp/ccam1ao5.o: In function `shutdown': syslog-node.c:(.text+0x39a): undefined reference to `app_shutdown' collect2: error: ld returned 1 exit status Those functions are available in the shared libraries.nm command output shows the same nsaboo@ubuntu:~/Desktop/Cproject$ nm syslog/syslog-ng-3.25.1/install/lib/syslog-ng/libdbparser.so | grep T | grep 'pattern_db_set_emit_func' 0000000000010d40 T pattern_db_set_emit_func nsaboo@ubuntu:~/Desktop/Cproject$ nm syslog/syslog-ng-3.25.1/install/lib/syslog-ng/libdbparser.so | grep T | grep 'pattern_db_new' 0000000000010ef0 T pattern_db_new nsaboo@ubuntu:~/Desktop/Cproject$ nm syslog/syslog-ng-3.25.1/install/lib/syslog-ng/libdbparser.so | grep T | grep 'pattern_db_reload_ruleset' 0000000000010cc0 T pattern_db_reload_ruleset nsaboo@ubuntu:~/Desktop/Cproject$ nm syslog/syslog-ng-3.25.1/install/lib/libsyslog-ng.so.0 | grep T | grep 'log_msg_set_value' 0000000000080df0 T log_msg_set_value 00000000000811d0 T __log_msg_set_value_by_name 0000000000081220 T log_msg_set_value_indirect nsaboo@ubuntu:~/Desktop/Cproject$ nm syslog/syslog-ng-3.25.1/install/lib/libsyslog-ng.so | grep T | grep 'msg_init' 0000000000051990 T msg_init nsaboo@ubuntu:~/Desktop/Cproject$ nm syslog/syslog-ng-3.25.1/install/lib/libsyslog-ng.so | grep T | grep 'cfg_new' 0000000000039200 T cfg_new 0000000000039360 T cfg_new_snippet 0000000000039390 T cfg_new_subordinate I understand this is the C linking error where I am including all the dynamic shared libraries and header files, but not able to figure out what is causing this issue. Can someone please help me to determine what am I missing here ? Thanks, Nitish

1 1

syslog-ng 3.26 Debian/Ubuntu packages (OBS)
by Laszlo Budai 05 Mar '20

05 Mar '20

Hi, syslog-ng 3.26.1[1] packages are available in OBS repositories[2, 3]. List of supported OSs: * Debian 8.0 * Debian 9.0 * Debian 10.0 * Raspbian 9.0 * Raspbian 10.0 * Ubuntu 14.04 * Ubuntu 16.04 * Ubuntu 18.04 * Ubuntu 18.10 * Ubuntu 19.04 * Ubuntu 19.10 Install ------- example: Debian 9.0 1. get release key wget -qO - http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… | sudo apt-key add - 2. add repo to APT sources eg.: /etc/apt/sources.list.d/syslog-ng-obs.list deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… ./ Then `apt-get update` and `apt-get install syslog-ng-core` Links -------- [1] https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.26.1 [2] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… [3] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… regards, Laszlo Budai

1 0

syslog-ng-3.26.1
by Laszlo Budai 05 Mar '20

05 Mar '20

3.26.1 Highlights * file source: Added a new option to multi-line file sources: multi-line-timeout() After waiting multi-line-timeout() seconds without reading new data from the file, the last (potentially partial) message will be flushed and sent through the pipeline as a LogMessage. Since the multi-line file source detects the end of a message after finding the beginning of the subsequent message (indented or no-garbage/suffix mode), this option can be used to flush the last multi-line message in the file after a multi-line-timeout()-second timeout. There is no default value, i.e. this timeout needs to be explicitly configured. Example config: file("/some/folder/events" multi-line-mode("prefix-garbage") multi-line-prefix('^EVENT: ') multi-line-timeout(10) flags("no-parse") ); (#2963<https://github.com/syslog-ng/syslog-ng/pull/2963>) * python-http-header: Added this new plugin, which makes it possible for users to implement HTTP header plugins in Python language. It is built on top of signal-slot mechanism: currently HTTP module defines only one signal, that is signal_http_header_request and python-http-header plugin implements a python binding for this signal. This means that when the signal_http_header_request signal is emitted then the connected slot executes the Python code. The Python interface is: def get_headers(self, body, headers): It should return string List. The headers that will be appended to the request's header. When the plugin fails, http module won't try to send the http request without the header items by default. If you want http module to trying to send the request without these headers, just disable mark-errors-as-critical() function. Original code was written by Ferenc Sipos. Example config, click to expand! (#3123<https://github.com/syslog-ng/syslog-ng/pull/3123>) * azure-auth-header: Added this new plugin, which generates authorization header for applications connecting to Azure. It can be used as a building block in higher level SCLs. Implemented as a signal-slot plugin. Example config, click to expand! (#3078<https://github.com/syslog-ng/syslog-ng/pull/3078>) * python: From now on users can specify a persist name template from python code. @staticmethod def generate_persist_name(options): return options["file_name"] * Usage of this function is necessary, when one python destination is used multiple times in one config. * Persist name from config takes precedence over generate_persist_name. * Persist name is exposed through self.persist_name. (#3016<https://github.com/syslog-ng/syslog-ng/pull/3016>) Features * set-severity(): Added this new rewrite rule for changing message severity. It receives a template and sets message severity by evaluating the template. Numerical and textual severity levels are both supported. Examples: rewrite { set-severity("info"); set-severity("6"); set-severity("${.json.severity}"); }; (#3115<https://github.com/syslog-ng/syslog-ng/pull/3115>) * $(list-search): Added a new template function, which returns the first index of a pattern in a list. Starts the search at start_index. 0 based. If not found, returns empty string. Usage: $(list-search --mode MODE <pattern> ${list}) Where mode can be: literal (default), prefix, substring, glob, pcre. Add --start-index <index> to change the start index. (#2955<https://github.com/syslog-ng/syslog-ng/pull/2955>) * config version: Made the config version check of the configuration more liberal by accepting version numbers that had no changes relative to the current version. This means that if you are running 3.26 and the last semantic change in the configuration was 3.22, then anything between 3.22 and 3.26 (inclusive) is accepted by syslog-ng without a warning at startup. (#3074<https://github.com/syslog-ng/syslog-ng/pull/3074>) * $SEVERITY instead of $LEVEL: syslog-ng now follows the RFC3164 trend of using severity instead of level to refer to the severity of the message that is used in the template language ($SEVERITY), filter expressions (severity()) and so on. (#3128<https://github.com/syslog-ng/syslog-ng/pull/3128>) * http: Added ssl_version("tlsv1_3") and ssl_version("no-tlsv13") options to respectively force and disable TLSv1.3. (#3063<https://github.com/syslog-ng/syslog-ng/pull/3063>) * scl: Improved error message at init, when an scl is missing a dependency. (#3015<https://github.com/syslog-ng/syslog-ng/pull/3015>) * geoip2: Added template() option as an alias for the positional argument string, to match the grammar convention. (#3051<https://github.com/syslog-ng/syslog-ng/pull/3051>) * loggly: Added transport() option, so users can now use it with tls (or any network() supported transport). (#3149<https://github.com/syslog-ng/syslog-ng/pull/3149>) * config-option-database: Added support for parser, diskq and hook-commands blocks. (#3029<https://github.com/syslog-ng/syslog-ng/pull/3029>) Bugfixes * configure.ac: Fixed gethostbyname() function location detection (#3135<https://github.com/syslog-ng/syslog-ng/pull/3135>) * http: Fixed a crash, when workers() was set to 0. We do not allow nonnegative values anymore. (#3116<https://github.com/syslog-ng/syslog-ng/pull/3116>) * snmp-dest: engine-id() option now handles 5 to 32 characters, instead of the strict 10 before. (#3058<https://github.com/syslog-ng/syslog-ng/pull/3058>) * http: Fixed handling of ssl-version() option, which was ignored before. Prior this fix, these values of ssl-version in http destination were ignored by syslog-ng: tlsv1_0, tlsv1_1, tlsv1_2, tlsv1_3. (#3083<https://github.com/syslog-ng/syslog-ng/pull/3083>) * network sources: Added workaround for a TLS 1.3 bug to prevent data loss. Due to a bug in the OpenSSL TLS 1.3 implementation (openssl/openssl#10880<https://github.com/openssl/openssl/issues/10880>), it is possible to lose messages when one-way communication protocols are used, - such as the syslog protocol over TLS (RFC 5425<https://tools.ietf.org/html/rfc5425>, RFC 6587<https://tools.ietf.org/html/rfc6587>) - and the connection is closed by the client right after sending data. The bug is in the TLS 1.3 session ticket handling logic of OpenSSL. To prevent such data loss, we've disabled TLS 1.3 session tickets in all syslog-ng network sources. Tickets are used for session resumption, which is currently not supported by syslog-ng. The loggen testing tool also received some bugfixes (#3064<https://github.com/syslog-ng/syslog-ng/pull/3064>), which reduce the likelihood of data loss if the target of loggen has not turned off session tickets. If you're sending logs to third-party OpenSSL-based TLS 1.3 collectors, we recommend turning session tickets off in those applications as well until the OpenSSL bug is fixed. (#3082<https://github.com/syslog-ng/syslog-ng/pull/3082>) * cmake: Now we install loggen headers, as we do with autotools. (#3067<https://github.com/syslog-ng/syslog-ng/pull/3067>) * graylog2, format-gelf: Fixed sending empty message, when ${PID} is not set. Also added a default value "-" to empty short_message and host as they are mandatory fields. (#3112<https://github.com/syslog-ng/syslog-ng/pull/3112>) * loggen: fix dependency error with cmake + openssl from nonstandard location (#3062<https://github.com/syslog-ng/syslog-ng/pull/3062>) * config-option-database: Fixed reading 'grammar' and 'parser' files on 'POSIX' environment (#3125<https://github.com/syslog-ng/syslog-ng/pull/3125>) * file source: Fixed file source not able to process new message after log-msg-size() increase. (#3075<https://github.com/syslog-ng/syslog-ng/pull/3075>) * checkpoint parser: Fixed parsing ISO timestamp. (#3056<https://github.com/syslog-ng/syslog-ng/pull/3056>) * secret-storage: Fixed some cases, where diagnostical logs were truncated. (#3141<https://github.com/syslog-ng/syslog-ng/pull/3141>) * loggen, dqtool: Fixed a crash, when writing error/debug message or relocating qfile. (#3069<https://github.com/syslog-ng/syslog-ng/pull/3069>) * build: Fixed a compatibility related build error on Solaris 11. (#3070<https://github.com/syslog-ng/syslog-ng/pull/3070>) * loggen: Fixed address resolution when only loopback interface was configured. (#3048<https://github.com/syslog-ng/syslog-ng/pull/3048>) Packaging * scl: Moved scl files to the core package. (#2979<https://github.com/syslog-ng/syslog-ng/pull/2979>) * RHEL: Now we include the packaging/rhel/ folder in our release tarball. (#3071<https://github.com/syslog-ng/syslog-ng/pull/3071>) * RHEL 8 / CentOS 8: Added RHEL 8 / CentOS 8 support to syslog-ng.spec (#3034<https://github.com/syslog-ng/syslog-ng/pull/3034>) Notes to developers * signal-slot-connector: Introduced a generic event handler interface for syslog-ng modules. * The concept is simple: * There is a SignalSlotConnector which stores Signal - Slot connections * Signal : Slot = 1 : N, so multiple slots can be assigned to the same Signal. * When a Signal is emitted, the connected Slots are executed. * Signals are string literals and each module can define its own signals. * Interface/protocol between signals and slots: * A macro (SIGNAL) can be used for defining signals as string literals: SIGNAL(module_name, signal, signal_parameter_type) The parameter type is for expressing a kind of contract between signals and slots. * Usage: #define signal_cfg_reloaded SIGNAL(config, reloaded, GlobalConfig) the generated literal: "config::signal_reloaded(GlobalConfig *)" * emit passes the argument to the slots connected to the emitted signal. (#3043<https://github.com/syslog-ng/syslog-ng/pull/3043>) * http: Defined the Signal interface for HTTP - with one signal at this time. What's in the changeset? * List ADT (abstract data type for list implementations) added to lib. * Interface: * append * foreach * is_empty * remove_all * Implemented the List ADT in http module with struct curl_slist for storing the headers. * HTTP signal(s): * Currently only one signal is added, header_request. Note, that the license for http-signals.h is LGPL . (#3044<https://github.com/syslog-ng/syslog-ng/pull/3044>) * example-modules: Added example http slot plugin. This plugin is an example plugin that implements a slot for a HTTP signal (signal_http_header_request). When the plugin is attached, it CONNECT itself to the signal. When the signal is emitted by the http module, the slot is executed and append the header to the http headers. header is set in the config file. Example config, click to expand! (#3093<https://github.com/syslog-ng/syslog-ng/pull/3093>) * NEWS.md: From now on, for every PR that we want to include in the newsfile, we must create the news entry with the PR itself. See news/README.md. (#3066<https://github.com/syslog-ng/syslog-ng/pull/3066>) * snmp test in Light: Added snmp destination tests in the Light test framework. These tests requires snmptrapd as an external dependency. If you don't want to run this test, you can use the pytest's marker discovery feature: python -m pytest ... -m 'not snmp' The tests are run by syslog-ng's Travis workflow. (#3126<https://github.com/syslog-ng/syslog-ng/pull/3126>) * cmake: Added add_module function to cmake to normalize CMakeLists.txt files for modules. (#3106<https://github.com/syslog-ng/syslog-ng/pull/3106>) * dbld: Introduced syslog-ng-kira as a new CI image (#3125<https://github.com/syslog-ng/syslog-ng/pull/3125>) * FunctionalTests: Functional tests are now Python3 compatible (#3144<https://github.com/syslog-ng/syslog-ng/pull/3144>) * dbld: Added Ubuntu 19.10 and 20.04 (#3091<https://github.com/syslog-ng/syslog-ng/pull/3091>) * dbld: Added option to customize shell command. With this change, it is possible to override the option with rules.conf, while keeping the default behaviour. The simplest example: use existing images, start a new one if there is none. (use docker rm manually if you want to update) DOCKER_SHELL=$(DOCKER) inspect $* > /dev/null 2>&1; \ if [ $$? -eq 0 ]; then \ $(DOCKER) start -ia $*; \ else \ $(DOCKER) run $(DOCKER_RUN_ARGS) -ti --name $* balabit/syslog-ng-$* /source/dbld/shell; \ fi (#3038<https://github.com/syslog-ng/syslog-ng/pull/3038>) Other changes * python: Added --with-python3-home configure option to use a hard-coded PYTHONHOME for Python-based plugins. This can be useful when a Python interpreter is bundled with syslog-ng. Relocation is supported, for example: --with-python3-home='${exec_prefix}' (#3134<https://github.com/syslog-ng/syslog-ng/pull/3134>) * afmongodb: Removed the support of deprecated legacy configurations (#3092<https://github.com/syslog-ng/syslog-ng/pull/3092>) * http: use-system-cert-store() now autodetects the system provided cert-store (#3086<https://github.com/syslog-ng/syslog-ng/pull/3086>) * doc: Added manual page for persist-tool. (#3072<https://github.com/syslog-ng/syslog-ng/pull/3072>) Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Andras Mitzki, Antal Nemes, Attila Szakacs, Balazs Scheidler, Dylan Perry, Gabor Nagy, Laszlo Budai, Laszlo Szemere, László Várady, Norbert Takacs, Peter Czanik, Péter Kókai, Romain Tartière, Tomáš Mózes.

1 0

Spurious path, logfile not created; path=
by Pal, Laszlo 04 Mar '20

04 Mar '20

Hi, For one of my hosts, I can see lots of these messages *Spurious path, logfile not created; path=* What does it mean exactly? I'm creating files with this macro file("/var/log/netlog/unix/${HOST}/${YEAR}/${MONTH}/${HOST}-${YEAR}${MONTH}${DAY}.log" and even for this host, I have all the logs regardless of this message I also have messages for the same host like this Resource temporarily unavailable (11) Here is some more details may help to find out the reasons behind this - issue started 9th February (I have a total of 160K entries like this) - the filename/path was incorrect during the whole event 2020/02/servername-20200210.log - on 29th the server gone south by consuming lots of CPU and disappeared from the network, console was frozen, so we had to reset the vm The host running an old syslog-ng PE (syslog-ng-premium-edition 4 LTS (4.0.5a) Installer-Version: 4.0.5a Revision: ssh+git://ganesa@git.balabit //var/scm/git/syslog-ng/syslog-ng-pe--mainline--4.0#master#457ec2f494a46d62ecf8cd938f12f02cd0ae9e63) on RHEL5 Log sources are simple plain text files contains tomcat and other web server logs I have a twin-host with the exact same config and log sources, but I never seen messages like this from that one Do you have any idea? To me it looks very mysterious Thanks Laszlo

4 5

syslog driver: IP instead of hostname
by Alexandre Santos 27 Feb '20

27 Feb '20

Hello, I have the following a syslog-ng server and a syslog-ng client, which configurations I am sending in attachment. I am using the syslog driver in order to have full compatibility with RFC5424. I want to use the IP address and not the hostname, but I keep seeing the hostname in tcpdump: [root@tests tests]# tcpdump -A -i virbr0 port 60514 or 514 tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on virbr0, link-type EN10MB (Ethernet), capture size 262144 bytes 18:30:09.810757 IP 192.168.122.11.34512 > tests.syslog: SYSLOG local0.info, length: 100 E.....@.@..K..z...z......l..<134>1 2020-02-26T18:30:09+00:00 localhost root 9519 - - This is a local0 info buffer filler string and in logfile: <134>1 2020-02-26T18:30:09+00:00 localhost root 9519 - - This is a local0 info buffer filler string Can you help me? Thanks in advance, Alex

2 2

system() vs systemd-journal()
by Alexandre Santos 27 Feb '20

27 Feb '20

Hi. I am using syslog-ng 3.19 in a Linux system. My sources configuration is: source s_src { system(); internal(); }; I would like to stack traces to end up in another facility than local0, which is what is happening with the configuration above. This happens because local0 is default facility when SYSLOG_FACILITY entry of journald is not defined. So I changed my sources configuration to: source s_src { internal(); systemd-journal(default-facility(local1)); }; And now the stacktraces are going to local1 facility. *Is there any difference between system() and systemd-journal() when using Linux? I am going to lose any logs?* Thanks in advance, Alex *BTW:* *source s_src { system(default-facility(local1)); internal();};* *Does not gives any error, but it does not have the desired effect.*

3 2

Re: [syslog-ng] network logging
by SZIGETVÁRI János 25 Feb '20

25 Feb '20

Hi Anatoly, I appreciate the trust you bestowed to me by sending me the bundle. >From syslog-ng perspective your setup looks all sane, and I couldn't spot any problems. Upon checking the debug logs, I saw, that syslog-ng did not report either of the incoming UDP packages as "Incoming messages" even though they were recorded in the pcap file. Given that SELinux is disabled, I and your firewall rules are all ALLOW, there are very few things that could prevent the reception of the messages. That led me to suspect a networking problem: a routing issue. You seem to have two interfaces active: A and B. A has a single IP configured, and B has two. Only two locally connected networks are present in the routing table, and a default route, that points to a host located in the subnet that interface A is connected to. The problem is that the packets arrive to interface B, and the IP address they arrive from should be in the direction of interface A, at least from routing perspective. Due to this the kernel will silently discard these packets. In order to fix this I would recommend you to review your routing table, or reconfigure your setup so that these packets arrive to interface A's configured IP address. If you want to verify this theory of mine, I can point you to the following article on how to make the kernel log these unroutable packets: https://www.cyberciti.biz/faq/linux-log-suspicious-martian-packets-un-routa… Best Regards, János -- Janos SZIGETVARI RHCE, License no. 150-053-692 <https://www.redhat.com/rhtapps/verify/?certId=150-053-692> LinkedIn: linkedin.com/in/janosszigetvari __@__˚V˚ Make the switch to open (source) applications, protocols, formats now: - windows -> Linux, iexplore -> Firefox, msoffice -> LibreOffice - msn -> jabber protocol (Pidgin, Google Talk) - mp3 -> ogg, wmv -> ogg, jpg -> png, doc/xls/ppt -> odt/ods/odp Anatoly Pugachev <matorola(a)gmail.com> ezt írta (időpont: 2020. febr. 25., K, 10:58): > On Wed, Feb 19, 2020 at 6:46 PM SZIGETVÁRI János <jszigetvari(a)gmail.com> > wrote: > > > > Hi Anatoly, > > > > Thank you for the debug log! > > It clearly shows that syslog-ng does not get the remote log at all. > > Could you please run the syslog-ng-debun script with the following > parameters, and send me the resulting tgz file? > > > > # syslog-ng-debun -r -P "port 3514" -D "-Fedv" -t 90 > > > > This will run syslog-ng in debug mode for 90 seconds, and will also > create a network traffic dump of the syslog traffic. Please try sending the > test logs from the other VM while it is running. > > > Janos, > > sorry for the delay, please see attached files. > > Thanks. > > tcpdump command was: > > localhost# tcpdump -n -w udp-3514.pcap -i ens224 udp and port 3514 > > this captured 2 packets coming from 2 different machines, which was run : > > vm1# logger -d -i -n 172.16.3.50 -P 3514 "test from vm1" > vm2# logger -d -i -n 172.16.3.50 -P 3514 "test from vm2" >

1 0

Re: [syslog-ng] Warnings and error while loading default.xml in syslog-ng-3.25.1
by Nitish Saboo 21 Feb '20

21 Feb '20

+Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> On Mon, Feb 17, 2020 at 2:37 PM Nitish Saboo <nitish.saboo55(a)gmail.com> wrote: > Hi Attila, > > 1,2) The issue is that you used "proxysg" in two different rulesets. I > believe syslog-ng could handle this, but we do not handle it right now. > >> But the same default.xml file is working fine in version > syslog-ng-3,6,2 and syslog-ng-3.7.1 where we have same program in different > rulesets.The behavior in version syslog-ng-3,6,2 and syslog-ng-3.7.1 is the > expected behavior or is the current behavior in syslog-ng-3.25.1 version is > having a Bug ? > > 3/4) You can merge the two rulesets, if it is okay for you: > >>Merging the rulesets is not possible at this moment. > Do we have any workaround for this scenario other than merging the > rulesets? > When can we expect a fix for this in near future ? > > Thanks, > Nitish > > On Mon, Feb 17, 2020 at 2:23 PM Attila Szakacs (aszakacs) < > Attila.Szakacs(a)oneidentity.com> wrote: > >> Hi! >> >> 1,2) The issue is that you used "proxysg" in two different rulesets. I >> believe syslog-ng could handle this, but we do not handle it right now. >> 3/4) You can merge the two rulesets, if it is okay for you: >> >> <?xml version='1.0' encoding='UTF-8'?> >> <patterndb version="4"> >> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description=""> >> <pattern>proxysg</pattern> >> <pattern>ProxySG</pattern> >> <rules> >> <rule id="f1e2bfd7bb85402a88d0b732821a0f94"> >> <patterns> >> <pattern>foo</pattern> >> </patterns> >> </rule> >> <rule id="a681963842014480a83a2a2e38875439"> >> <patterns> >> <pattern>anything</pattern> >> </patterns> >> </rule> >> <rule id="bb169f917216467985cc16e28015f5fa"> >> <patterns> >> <pattern>bar</pattern> >> </patterns> >> </rule> >> <rule id="94d4a0c324c8-44a88cf3d4640477d35e"> >> <patterns> >> <pattern>something</pattern> >> </patterns> >> </rule> >> </rules> >> </ruleset> >> </patterndb> >> >> Regards, >> Attila >> >> ------------------------------ >> *From:* syslog-ng <syslog-ng-bounces(a)lists.balabit.hu> on behalf of >> Nitish Saboo <nitish.saboo55(a)gmail.com> >> *Sent:* Monday, February 17, 2020 6:30 AM >> *To:* Syslog-ng users' and developers' mailing list < >> syslog-ng(a)lists.balabit.hu> >> *Subject:* Re: [syslog-ng] Warnings and error while loading default.xml >> in syslog-ng-3.25.1 >> >> CAUTION: This email originated from outside of the organization. Do not >> follow guidance, click links, or open attachments unless you recognize the >> sender and know the content is safe. >> >> Hi Evan, >> >> Apologies for the confusion but I did close the rule tag in my >> default.xml , missed adding it here. >> The error does not seem to be related to improper closing of tags. >> >> This is how it looks : >> >> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description=""> >> <pattern>proxysg</pattern> >> <rules> >> <rule id="f1e2bfd7bb85402a88d0b732821a0f94"> >> <patterns> >> <pattern>foo</pattern> >> </patterns> >> </rule> >> <rule id="a681963842014480a83a2a2e38875439"> >> <patterns> >> <pattern>anything</pattern> >> </patterns> >> </rule> >> </rules> >> </ruleset> >> <ruleset id="17fae6edff32a53f9f294ab21240fc2641e7a4db" description=""> >> <pattern>ProxySG</pattern> >> <pattern>proxysg</pattern> >> <rules> >> <rule id="bb169f917216467985cc16e28015f5fa"> >> <patterns> >> <pattern>bar</pattern> >> </patterns> >> </rule> >> <rule id="94d4a0c324c8-44a88cf3d4640477d35e"> >> <patterns> >> <pattern>something</pattern> >> </patterns> >> </rule> >> </rules> >> </ruleset> >> >> >> I am getting the following error message: >> >> 2020-02-13T10:47:29.631090] Error parsing pattern database file; >> filename='/home/nsaboo/abc/default.xml', >> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with >> mismatching program name sets, program=proxysg'. >> >> My hunch is the issue is related to merging of two rulesets but I am not >> able to understand why is there mismatching of program names. >> >> 1) Can someone please help me understand the issue here ? >> >> 2) Is the issue seen because a ruleset has multiple programs in it or is >> it because the same program 'proxysg' is being used in different rulesets ? >> >> 3) From the above snippet of default.xml, what changes can I make into >> default.xml to avoid the error ? >> >> 4) Is there a workaround for this issue ? >> >> Thanks, >> Nitish >> >> >> On Sun, Feb 16, 2020 at 12:40 AM Evan Rempel <erempel(a)uvic.ca> wrote: >> >> I'm not exactly sure what is or is not permitted in the pattern database >> but I two comments. >> >> 1. you need end your rule tag and your rules tag before you start a new >> ruleset tag. >> >> 2. What I do in my pattern database is of the form. >> >> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description=""> >> <pattern>proxysg</pattern> >> <rules> >> <rule id="f582419b3baa42d4a57e42b89704e38c"> >> <patterns> >> <pattern>foo</pattern> >> </patterns> >> </rule> >> <rule id="bb169f917216467985cc16e28015f5fa"> >> <patterns> >> <pattern>bar</pattern> >> </patterns> >> </rules> >> </ruleset> >> >> >> Note: >> 1. the closing tag of </rule> before a new starting tag of <rule> >> 2. Multiple "rule" entries inside the the "rules" entry. >> 3. the closing tag of </rule> before the closing tag of </rules> >> 4. the closing tag of </rules> before the closing tag of </ruleset> >> >> I hope that helps. >> >> Evan. >> >> On 2/15/20 12:43 AM, Nitish Saboo wrote: >> >> Hi, >> >> After debugging further into the issue looks like there was a fix for >> patterndb rule clash in syslog-ng-3.8 and this is the commit-id >> '12cd960c8f47260b0b0d4154b096994d66fe345' >> for the fix. And for this reason I am getting the following error for >> same default.xml in syslog-ng-3.25.1 version and not in syslog-ng3.6.2 and >> syslog-ng3.7.1. >> >> 2020-02-13T10:47:29.631090] Error parsing pattern database file; >> filename='/home/nsaboo/abc/default.xml', >> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with >> mismatching program name sets, program=proxysg'. >> >> Snippet from default.xml >> ========================== >> >> <ruleset id="f582419b3baa42d4a57e42b89704e38c" description=""> >> <pattern>proxysg</pattern> >> <rules> >> <rule id="f582419b3baa42d4a57e42b89704e38c"> >> <patterns> >> <pattern>foo</pattern> >> </patterns> >> >> <ruleset id="8d633c824e844a559088d803464e507a" description=""> >> <pattern>ProxySG</pattern> >> <pattern>proxysg</pattern> >> <rules> >> <rule id="bb169f917216467985cc16e28015f5fa"> >> <patterns> >> <pattern>bar</pattern> >> </patterns> >> >> I am not able to understand the error message clearly. >> >> 1) Can someone please help me understand the issue here ? >> >> 2) Is the issue seen because a ruleset has multiple programs in it or is >> it because the same program 'proxysg' is being used in different rulesets ? >> >> 3) From the above snippet of default.xml, what changes can I make into >> default.xml to avoid the error ? >> >> 4) Is there a workaround for this issue ? >> >> Thanks, >> Nitish >> >> On Fri, Feb 14, 2020 at 2:40 PM Nitish Saboo <nitish.saboo55(a)gmail.com> >> wrote: >> >> Hi Attila, >> >> Thanks for your response. >> >> And what about the following error: >> >> 2020-02-13T10:47:29.631090] Error parsing pattern database file; >> filename='/home/nsaboo/abc/default.xml', >> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with >> mismatching program name sets, program=proxysg'. >> >> The same default.xml file was getting loaded correctly in syslog-ng-3.6.2 >> and syslog-ng-3.7.1 but getting following error while loading same >> default.xml in syslog-ng3.25.1 >> >> I came across a similar issue on githib ' >> https://github.com/syslog-ng/syslog-ng/issues/2763 >> <https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.co…>' >> .I see the issue is still in open state.Is there a workaround for >> this issue? >> >> Thanks, >> Nitish >> >> On Fri, Feb 14, 2020 at 1:12 PM Attila Szakacs (aszakacs) < >> Attila.Szakacs(a)oneidentity.com> wrote: >> >> Hi! >> >> WARNING: due to a bug in versions before syslog-ng 3.8numeric comparison >> operators like '!=' in filter expressions were evaluated as string >> operators. This is fixed in syslog-ng 3.8. As we are operating in >> compatibility mode, syslog-ng will exhibit the buggy behaviour as previous >> versions until you bump the @version value in your configuration file; >> >> ^^^ This refers to the syslog-ng.conf file version. >> >> The correct way to resolve it, and fix the buggy behavior of != and ==, >> should be to change the != operators between strings to neq in your filters. >> >> Regards, >> Attila >> ------------------------------ >> *From:* syslog-ng <syslog-ng-bounces(a)lists.balabit.hu> on behalf of >> Nitish Saboo <nitish.saboo55(a)gmail.com> >> *Sent:* Thursday, February 13, 2020 12:17 PM >> *To:* Syslog-ng users' and developers' mailing list < >> syslog-ng(a)lists.balabit.hu> >> *Subject:* [syslog-ng] Warnings and error while loading default.xml in >> syslog-ng-3.25.1 >> >> Hi, >> >> I am using syslog-ng version 3.25.1.Getting following Warnings and error >> while initialising syslog-ng engine: >> >> [2020-02-13T10:47:29.627899] WARNING: due to a bug in versions before >> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions >> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we >> are operating in compatibility mode, syslog-ng will exhibit the buggy >> behaviour as previous versions until you bump the @version value in your >> configuration file; >> [2020-02-13T10:47:29.627968] WARNING: due to a bug in versions before >> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions >> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we >> are operating in compatibility mode, syslog-ng will exhibit the buggy >> behaviour as previous versions until you bump the @version value in your >> configuration file; >> [2020-02-13T10:47:29.628059] WARNING: due to a bug in versions before >> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions >> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we >> are operating in compatibility mode, syslog-ng will exhibit the buggy >> behaviour as previous versions until you bump the @version value in your >> configuration file; >> [2020-02-13T10:47:29.631090] Error parsing pattern database file; >> filename='/opt/tap-parsing/patterns/default.xml', >> error='/opt/tap-parsing/patterns/default.xml:17274:22: Joining rulesets >> with mismatching program name sets, program=proxysg' >> >> >> 1)For the following warnings, to which version I have to bump up the >> configuration file ? >> >> 2020-02-13T10:47:29.627899] WARNING: due to a bug in versions before >> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions >> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we >> are operating in compatibility mode, syslog-ng will exhibit the buggy >> behaviour as previous versions until you bump the @version value in your >> configuration file; >> [2020-02-13T10:47:29.627968] WARNING: due to a bug in versions before >> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions >> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we >> are operating in compatibility mode, syslog-ng will exhibit the buggy >> behaviour as previous versions until you bump the @version value in your >> configuration file; >> [2020-02-13T10:47:29.628059] WARNING: due to a bug in versions before >> syslog-ng 3.8numeric comparison operators like '!=' in filter expressions >> were evaluated as string operators. This is fixed in syslog-ng 3.8. As we >> are operating in compatibility mode, syslog-ng will exhibit the buggy >> behaviour as previous versions until you bump the @version value in your >> configuration file; >> >> Currrently the configuration version is the following: >> >> configuration = cfg_new(0x0302) >> >> Do I have to change it to '0x0319' as defined in 'lib/versioning.h' ? >> >> 2)The same default.xml file was getting loaded correctly in >> syslog-ng-3.6.2 and syslog-ng-3.7.1 but getting following error while >> loading same default.xml in syslog-ng3.25.1 >> >> >> 2020-02-13T10:47:29.631090] Error parsing pattern database file; >> filename='/home/nsaboo/abc/default.xml', >> error='/home/nsaboo/abc/default.xml:17274:22: Joining rulesets with >> mismatching program name sets, program=proxysg'. >> >> What can be the reason for this error ? >> >> >> Thanks, >> Nitish >> >> >> >> ______________________________________________________________________________ >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >> <https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.bal…> >> Documentation: >> http://www.balabit.com/support/documentation/?product=syslog-ng >> <https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabi…> >> FAQ: http://www.balabit.com/wiki/syslog-ng-faq >> <https://nam05.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabi…> >> >>

4 7

Java Destination Method Documentation
by Jamal Habash 20 Feb '20

20 Feb '20

Hey All, I am interested in developing a Java destination for syslog-ng. I have read the documentation which states "To interface with syslog-ng, you will need to extend the TextLogDestination or StructuredLogDestination". There is no documentation or code comments for either of these classes. 1) What is the intended use-case for TextLogDestination? 2) What is the intended use-case for StructuredLogDestination? Additionally, as there are no comments in the code, it is unclear what the abstract methods in these classes are meant to do. For example, each class has the same abstract methods called open(), close(), isOpened(), getNameUniqOptions(). 3) What is the purpose of these methods? 4) When will they be invoked by syslog-ng? It is difficult to write a valid implementation without knowing the purpose of each method. Is there any documentation for this? Thanks in advance! J

2 2

network logging
by Anatoly Pugachev 19 Feb '20

19 Feb '20

Hello! Can someone help me with network logging with syslog-ng : # rpm -q syslog-ng syslog-ng-3.25.1.239.g0535e8a-1.el7.x86_64 # rpm -qf /etc/os-release centos-release-7-7.1908.0.el7.centos.x86_64 server is centos 7 x86_64 vm /etc/syslog-ng/syslog-ng.conf is in the default configuration (no changes made from rpm package installed), have additional configuration in : # cat conf.d/mtcaptive.conf source s_mtcaptive { network( ip("0.0.0.0") port(3514) transport("udp")); }; destination d_mtcaptive { file("/var/log/mt-captive-remote"); }; log { source(s_mtcaptive); destination(d_mtcaptive); }; If i try to log a test message with logger on a localhost (where syslog-ng runs): [root@localhost syslog-ng]# logger -d -i -n 172.16.3.50 -P 3514 "test from localhost" it does work and appears in /var/log/mt-captive-remote : $ tail -1 /var/log/mt-captive-remote Feb 19 06:15:01 172.16.3.50 root[11326]: test from localhost But if i try the same command on a nearby host (vm2, same L2 network, but different IP network), it does not logged, but udp packet with test messages is received by host: [root@vm2 ~]# logger -d -i -n 172.16.3.50 -P 3514 "test from vm2" back to syslog-ng host: # ngrep -d ens224 -n 10 "test from" "udp and port 3514" interface: ens224 (172.16.3.0/255.255.255.0) filter: ( udp and port 3514 ) and ((ip || ip6) || (vlan && (ip || ip6))) match: test from ########################################################################################################################################################################################################################## U 100.64.128.5:24046 -> 172.16.3.50:3514 #218 <5>Feb 19 06:20:27 root[31663]: test from vm2. #####################################################################################################^Cexit 319 received, 1 matched looking at socket status on syslog-ng host: # ss -unlp | grep 3514 UNCONN 0 0 *:3514 *:* users:(("syslog-ng",pid=11198,fd=23)) no firewall or iptables rules (default all ACCEPT). Can someone suggest why syslog-ng does not log remote messages to my destination file? Thanks.

2 3