- syslog-ng - lists.balabit.hu

Mutual TLS issues
by Matthew Pounsett 19 Sep '18

19 Sep '18

I'm trying to get mutual TLS validation working and seem to be having issues with the config. The situation I've got is that I'm trying to accept a stream from another organization. And so my preferred configuration involves two CA's.. mine and theirs. My syslog-ng server needs to validate their server certificate using their CA certificate, and their syslog-ng client needs to validate my server certificate using my CA certificate. I've set up a test using the server that will be production, and a handy VM I control. I assumed that it would be as simple as using the example configs from the manual [0], and anywhere a CA cert is involved just pick the appropriate one: store their CA on my server, and my CA on "their" server (the test VM). That didn't work. I've backed down to trying to get this working using all certificates signed by my CA, but I can't seem to make that work either. The error timing I'm getting doesn't quite match similar errors I can see reported online. If I don't have the correct CA cert on the client, then I see a connect, immediate disconnect, and a "SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca" error on the server. If I do have the CA cert on the client, I see a connect ... and then nothing until I try to send a message. Then I get the same error on the server and a logged "Certificate validation failed" error on the client referencing the server's cert, and a disconnect. Server logs: Sep 17 23:17:49 adm syslog-ng[31128]: Syslog connection accepted; fd='24', client='AF_INET(216.235.10.40:54063)', local='AF_INET(64.191.0.17:6514)' Sep 17 23:18:01 adm syslog-ng[31128]: SSL error while reading stream; tls_error='SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca' Sep 17 23:18:01 adm syslog-ng[31128]: I/O error occurred while reading; fd='24', error='Connection reset by peer (104)' Sep 17 23:18:01 adm syslog-ng[31128]: Syslog connection closed; fd='24', client='AF_INET(216.235.10.40:54063)', local='AF_INET(64.191.0.17:6514)' Client logs: Sep 17 23:17:48 ubuntu syslog-ng[4603]: syslog-ng starting up; version='3.13.2' Sep 17 23:17:48 ubuntu syslog-ng[4603]: Syslog connection established; fd='13', server='AF_INET(64.191.0.17:6514)', local='AF_INET(0.0.0.0:0)' Sep 17 23:18:00 ubuntu syslog-ng[4603]: Certificate validation failed; subject='emailAddress=xxx(a)dns-oarc.net, CN=64.191.0.17, O=OARC\, Inc., L=Indianapolis, ST=Indiana, C=US', issuer='emailAddress=xxx(a)dns-oarc.net, CN=OARC Inc. Certificate Authority, O=OARC\, Inc., ST=Indiana, C=US', error='unable to get local issuer certificate', depth='0' Sep 17 23:18:00 ubuntu syslog-ng[4603]: SSL error while writing stream; tls_error='SSL routines:tls_process_server_certificate:certificate verify failed' Sep 17 23:18:00 ubuntu syslog-ng[4603]: I/O error occurred while writing; fd='13', error='Broken pipe (32)' Sep 17 23:18:00 ubuntu syslog-ng[4603]: Syslog connection broken; fd='13', server='AF_INET(64.191.0.17:6514)', time_reopen='60' Note the 12 second gap between startup and cert validation failure. The failure coincides with logging a message that should be sent from the client to the server. Server config: source tls_source { network(ip(64.191.0.17) port(6514) transport("tls") tls( key-file("/etc/ssl/private/64.191.0.17.key") cert-file("/etc/ssl/certs/64.191.0.17.crt") ca-dir("/etc/ssl/certs") ) ); }; Client config: destination d_tls { network("64.191.0.17" port(6514) transport("tls") tls( ca-dir("/etc/ssl/certs") key-file("/etc/ssl/private/216.235.10.40.key") cert-file("/etc/ssl/certs/216.235.10.40.crt") ) ); }; I have the CA cert stored as /etc/ssl/certs/server_ca.pem on both systems. Any idea what's wrong here? And, should I be able to get this to work using different CAs for the client and server certs? Thanks [0]: <https://www.syslog-ng.com/technical-documents/doc/ syslog-ng-open-source-edition/3.16/mutual-authentication- using-tls#TOPIC-956368>

2 3

@ as delimiter in patterndb parser
by joao reis 13 Sep '18

13 Sep '18

How can I specify @ as a delimiter in patterndb? I want to parse user and domain separated in email adresses. Thanks in advance.

2 1

Packet loss.
by vijay amruth 13 Sep '18

13 Sep '18

Hello everyone, we’ve been experiencing packet loss on syslog-ng collectors (these are open stack VMs), lately we have increased ingestion and that refelcted packet loss ( especially UDP). Tuning parameters didn’t help. If anyone here on the list has experienced a similar issue please share your resolution or work around. We are at 3.9.1, we are planning to update to 3.17.x for better data handling/other bug fixes. As an alternate approach we have increased the VM capacity ( no of VMs). If there is any that I missed, please point. Any thoughts? Thank you, Vijay Amrut. -- Thanks, Vijay Amrut.

3 4

Crash on syslog-ng 3.8.1
by xen rkar 12 Sep '18

12 Sep '18

Hi, Looking to see if this is a know issue, cause i have been unsucessful in find an exact match of the signature anywhere in the forums. Regards, rkar #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x76c5775c in __GI_abort () at abort.c:79 #2 0x76f537a8 in iv_fatal (fmt=0x76f750dc "iv_timer_unregister: timer index %d > %d") at ../../../../syslog-ng-3.8.1/lib/ivykis/src/iv_fatal.c:57 #3 0x76f53f4c in iv_timer_unregister (_t=0x17e5284) at ../../../../syslog-ng-3.8.1/lib/ivykis/src/iv_timer.c:305 #4 0x76f53fb8 in iv_run_timers (st=st@entry=0x17a50c0) at ../../../../syslog-ng-3.8.1/lib/ivykis/src/iv_timer.c:148 #5 0x76f55ad8 in iv_main () at ../../../../syslog-ng-3.8.1/lib/ivykis/src/iv_main_posix.c:107 #6 0x76f24524 in main_loop_run () at ../syslog-ng-3.8.1/lib/mainloop.c:497 #7 0x000113e0 in main (argc=70624, argv=0x7ea96054) at ../syslog-ng-3.8.1/syslog-ng/main.c:289

2 1

syslog-ng never frees RAM
by Jose Angel Santiago 10 Sep '18

10 Sep '18

Hi, I'm using syslog-ng 3.16.1 with wildcard-file as source (let's call it "syslog-agent), which sends log messages to another syslog-ng acting as a relay. I've noticed that syslog-agent instances RAM consumption keeps increasing until they leave no free memory in the cluster (each server has 64G of RAM). In my use case, new folders & files are created constantly under base-dir folder, but every 2 days obsolete folders & files are deleted. I assumed that syslog-ng would free some RAM every time those folders & files are deleted, but it doesn't, not even if I run a syslog-ng-ctl reload operation. log-fifo-size value is 345000, so I assume it can't be a buffer situation since 345000 messages can't occupy 40-50 GB of memory. I've performed the following test to reproduce the situation in small scale: - Launch a syslog-agent with a wildcard-file source reading from "/tmp/test/" base-dir. syslog-agent RAM usage is about 125M. - Run a simple script to create complex folder hierarchy under /tmp/test and some files with 5000 log messages to read from. - Wait until syslog-agent RAM usage gets 1GB - Stop script execution and wait until syslog-agent has send all logs to relay. - Delete everything under /tmp/test and execute syslog-ng-ctl reload operation - 24 hours after, syslog-agent RAM usage still is 1GB I've used heaptrack tool as a try to find a memory leak in syslog-ng, you can see in the attached image that iv_list_empty function in iv_list.h file is where most of the RAM usage is. How do I get syslog-ng to free RAM? Or is it a memory leak? Thanks in advance. -- | Jose Angel Santiago [image: Logo_signature2.png] <http://www.stratio.com/> Vía de las dos Castillas, 33, Ática 4, 3ª Planta 28224 Pozuelo de Alarcón, Madrid, Spain +34 918 286 473 | www.stratio.com <https://twitter.com/stratiobd> <https://www.linkedin.com/company/stratiobd> <https://www.youtube.com/c/StratioBD>

3 7

(no subject)
by R.Naveen Naidu 08 Sep '18

08 Sep '18

Hi, I need a help to choose the syslog-ng package for Linux. What are the things need to consider when choosing the latest syslog-ng to Linux? Actually, I wanted to cross-compile the syslog-ng and load into the customized Linux version *4.1.35rt41* The current latest syslog-ng version is* 3.17.2* from below site and planning to use this. https://github.com/balabit/syslog-ng/releases -- Thanks & Regards Naveen

3 8

problems compiling 3.17.2 on Ubuntu 16.0.4 with java
by Russell Fulton 04 Sep '18

04 Sep '18

I am trying to build syslog-ng with java and json support on Ubuntu 16.0.4. I can’t use prepackaged version because ubuntu have decided that one must not have both rsyslog (which is our corporate standard) and syslog-ng (which I need for its pattern matching in an application) on the same machine. Installing one package removes the other. As well as the standard build tools I have installed these packages via apt: jsoncpp, jre and gradle. Simply running ./configure ran through but did not find java (i.e. java and java-module were both set to no). I added —enable java and configure terminated complaining that it could not find java. I dug into the configure script and found that it was executing “which javac” and “javah” both of which failed. I changed JAVAC to “java” and the configure script then completed with both java and java module enabled. Running make then failed builder@secgendev02:~/src/syslog-ng-3.17.2$ make mkdir -p modules/java/syslog-ng-core/classes GEN modules/java/syslog-ng-core/classes/org/syslog_ng/LogMessage.class Unrecognized option: -d Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit. Makefile:20740: recipe for target 'modules/java/syslog-ng-core/classes/org/syslog_ng/LogMessage.class’ failed I am guessing that the version of java the jre package installed is not compatible with the configure script. builder@secgendev02:~/src/syslog-ng-3.17.2$ java -version openjdk version "1.8.0_181" OpenJDK Runtime Environment (build 1.8.0_181-8u181-b13-0ubuntu0.16.04.1-b13) OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode) I know that configure checks the java version so I am somewhat surprised… Any Ideas on what to try next? Russell

2 1

Running into http module error.
by vijay amruth 30 Aug '18

30 Aug '18

Hello everyone, Running into this error: [root@xxxxxxxx ~]# syslog-ng -e [2018-08-29T10:53:16.602437] Plugin module not found in 'module-path'; module-path='/usr/local/lib/syslog-ng', module='http' Where do I get this module or how do I fix this, any help is apprciated? -- Thanks, Vijay Amrut.

3 2

How to rewrite structured data - XML, Windows Events and format-json
by Jim Hendrick 28 Aug '18

28 Aug '18

I am struggling with a configuration to rewrite specific sections of a Windows Event log message and then send them along to splunk (I think json is probably the easiest for splunk to ingest). Many windows events contain extraneous text (e.g. "This event.....") that I would like to remove. I have been able to rewrite it using a regex "This event.*</Message>" and replacing that with "DEADBEEF</Message>" The problem is that I can only see this in a test file destination and not in one that uses format-json (I set this up from the syslog-ng WEF configuration guide) Has anyone done anything like this? I think my problem is that I don't understand how the message is changed from it's arrival through processing. My test can rewrite it within MSG or MESSAGE - but when I try to use the rewrite() in the log statement using the format-json destination, nothing seems to be changed. Tips? Thanks! Jim

2 1

Connect syslog-ng to HDFS
by Lee Keng Ket 27 Aug '18

27 Aug '18

Hi, I'm trying to connect syslog-ng 3.14.1 to HDFS to store the syslog messages. The syslog-ng can start without error, and it's able to write into local file. However, the log is not written to the HDFS. As there is no single error, I'm not sure how I should troubleshoot on this. I have installed the syslog-ng from this repo, https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng314/repo/epel-7/cza… Installed Packages syslog-ng.x86_64 3.14.1-4.el7.centos @czanik-syslog-ng314 syslog-ng-java.x86_64 3.14.1-4.el7.centos @czanik-syslog-ng314 syslog-ng-java-deps.noarch 1.0-2 @czanik-syslog-ng314 This is the message from /var/log/message: Log statistics; processed='src.internal(s_sys#0)=1', stamp='src.internal(s_sys#0)=1534491834', processed='destination(d_spol)=0', processed='destination(d_mlal)=0', processed='center(received)=2', processed='destination(d_mesg)=1', processed='destination(d_mail)=0', processed='destination(d_auth)=0', processed='destination(d_cron)=0', processed='destination(d_hdfs)=1', processed='center(queued)=3', queued='global(scratch_buffers_count)=0', processed='source(remote_log)=1', dropped='dst.java(d_hdfs#0,java_dst,hdfs,hdfs://x.x.x.x:25000,/user/syslog/$HOST-$DAY-$MONTH-$YEAR.log)=0', processed='dst.java(d_hdfs#0,java_dst,hdfs,hdfs://x.x.x.x:25000,/user/syslog/$HOST-$DAY-$MONTH-$YEAR.log)=1', queued='dst.java(d_hdfs#0,java_dst,hdfs,hdfs://x.x.x.x:25000,/user/syslog/$HOST-$DAY-$MONTH-$YEAR.log)=1', processed='global(payload_reallocs)=0', processed='src.journald(journal)=0', stamp='src.journald(journal)=0', processed='global(sdata_updates)=0', queued='global(scratch_buffers_bytes)=0', processed='destination(d_boot)=0', processed='destination(d_kern)=0', processed='source(s_sys)=1', processed='destination(remote)=1', processed='global(internal_queue_length)=0', processed='global(msg_clones)=0' Anyone has any idea how should I proceed the troubleshooting?

6 20