- syslog-ng - lists.balabit.hu

'network' Destination With Hostname Resolution (IPv4 vs IPv6)
by David Hauck 06 Oct '18

06 Oct '18

Hi, I have a question regarding how to specify a network destination when using a hostname when the hostname can be resolved to either IPv4 or IPv6. In particular what should be specified by the ip-protocol() parameter? There are some configuration scenarios and/or target installations that don't know a priori whether the DNS configuration will resolve to an IPv4 or IPv6 address. E.g., destination d_tcp6 { network( "myhost" port(514) transport(udp) ip-protocol(6 or 4 or ??) ); }; It seems like it would simple enough to have syslog-ng simply validate the resulting IP address string to determine which of ip-protocol(4) or ip-protocol(6) is actually needed. In fact, I would argue that specifying an IP address string (as the "<destination-address>" value) could result in the same determination (the address string necessarily unambiguously determines whether the reference is an IPv4 or an IPv6 address and I would think there is a 1-1 relationship between this determination and whether ip-protocol(4) or ip-protocol(6) is used - (in other words it would never make sense to have these mixed: "::1" and ip-protocol(4) would be invalid). Thanks, -David

3 12

pattern usage and optimization
by Joel Carnat 01 Oct '18

01 Oct '18

Hi, I want to parse my sshd logs to store information in a remote database. I already did it using logstash. But I just discovered syslog-ng can do such things using patterndb. I could manage to setup a few <pattern> but I have difficulties building generic rules. I end up with 4 or 5 rules where I can only deal with one or two using logstash. So I expect to be missing something with patterns :) Here's a log example: Disconnected from user joe 192.168.0.5 port 50121 Disconnected from invalid user www 192.168.0.7 port 6794 [preauth] Disconnected from authenticating user root 192.168.0.3 port 52591 [preauth] So I wrote those three patterns:  <pattern>@ESTRING:EVENT: from @user @ESTRING:USERNAME: @@ESTRING:IP: @port @NUMBER:PORT:@</pattern>  <pattern>@ESTRING:EVENT: from @invalid user @ESTRING:USERNAME: @@ESTRING:IP: @port @NUMBER:PORT:@@ANYSTRING:EXTRA:@</pattern>  <pattern>@ESTRING:EVENT: from @authenticating user @ESTRING:USERNAME: @@ESTRING:IP: @port @NUMBER:PORT:@@ANYSTRING:EXTRA:@</pattern> To me, those 3 lines can be described using a single expression this way : ("Disconnected from") ("user"|"invalid user"|"authenticating user") (username) (ip_host) port (ip_port)(empty|extra_stuff) Basically, the features I couldn't find are : - "match a defined string and affect to variable" - "match a string or another and affect to variable" - "match a string or EOL and affect to variable if not empty". Is it possible to have a single pattern that would lead to have: - EVENT = "Disconnected from" - METHOD = "user" | "invalid user" | "authenticating user" - USERNAME = <parsed username> - IP = <parsed ip address> - PORT = <parsed port number> - EXTRA = <empty> | <parsed extra information> Thanks for you help.

2 3

Undesirable behavior from Cisco parser?
by Nik Ambrosch 27 Sep '18

27 Sep '18

Recently I upgraded my centralized loghost from 3.9 -> 3.15 and I noticed that some of my cisco devices started being logged in an undesirable format... I don't want to enable the cisco parser because more than just cisco messages get delivered to this interface. Here are the relevant fields that have changed before/after the upgrade: syslog-ng 3.9, before upgrade --- ${FULLHOST}: "mydevice.com" ${PROGRAM}: "" message: "%CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for..." syslog-ng 3.15, before upgrade --- ${FULLHOST}: ":" ${PROGRAM}: "%CRYPTO-4-RECVD_PKT_INV_SPI" ${MSG}: "decaps: rec'd IPSEC packet has invalid spi for..." Is this unintended behavior or a bug? This particular device is a Cisco 3845 running ios 12.4(22)T4. Thanks in advance.

5 17

disk-buffer in elasticsearch2 destination loses messages if docker container is killed
by Jose Angel Santiago 25 Sep '18

25 Sep '18

Hi, I'm running syslog-ng (with an elasticsearch2 destination configured) within a docker container, and I'm trying to avoid loss of messages if I kill the docker container and I start it again. This is my scenary: - A service which produces 20 lines of log per second - A sislog-ng instance reading from a wildcard-file source (but actually it only reads logs from the above service, let's call it syslog-agent), which sends all logs to another syslog-ng instance (the one running in a docker container, let's call it syslog-relay) though a network destination. - The syslog-relay sends messages to an elasticsearch instance, with following configuration: options { chain-hostnames(no); use-dns(no); keep-hostname(yes); owner("syslog-ng"); group("stratio"); perm(0640); time-reap(30); mark-freq(10); stats-freq(0); bad-hostname("^gconfd$"); flush-lines(100); log-fifo-size(1000); }; *destination d_elastic_default_0 { elasticsearch2( cluster("myelastic") cluster-url("https://myelastic.logs:9200 <https://myelastic.logs:9200>") client_mode("https") index("default") type("log") flush-limit(1) disk-buffer( mem-buf-size(16M) disk-buf-size(16M) reliable(yes) dir("/syslog-ng/log") ) http-auth-type("clientcert") java-keystore-filepath("/etc/syslog-ng/certificates/syslog-relay.jks") java-keystore-password("XXXXXX") java-truststore-filepath("/etc/syslog-ng/certificates/ca-bundle.jks") java-truststore-password("XXXXXXXXXX") );};* - The dir "/syslog-ng/log" is mapped to a path "/tmp/buffer" from the host where the docker container is running, so when I kill the docker container, the buffer file is not lost. - I've set flush-limit to 1 because I thought that I may lost 1 message only as much. This architecture is working fine (flush-limit=1 makes very slow, but for this test is ok), but if I kill the syslog-relay docker container, wait 5 to 10 seconds and start it again from scratch, I can see that several hundreds of logs are missing in elasticsearch. I check it by stopping the logger service and letting syslog-ng agent & relay to finish the process enqueued messages. I can see in the syslog-agent stats that all logs messages have been processed, so it seems the problem is on the syslog-relay. Is this behaviour expected? If so, how can I protect against loss of messages in case of a syslog-relay docker container unexpected kill? Thanks in advance. -- | Jose Angel Santiago [image: Logo_signature2.png] <http://www.stratio.com/> Vía de las dos Castillas, 33, Ática 4, 3ª Planta 28224 Pozuelo de Alarcón, Madrid, Spain +34 918 286 473 | www.stratio.com <https://twitter.com/stratiobd> <https://www.linkedin.com/company/stratiobd> <https://www.youtube.com/c/StratioBD>

3 16

Syslog-ng missing few logs to write to local file after showing in tcpdump
by Harish Shetty 24 Sep '18

24 Sep '18

Hi All I have syslog-ng (syslog-ng-3.9.1-1.el6.x86_64) which is running on Red Hat 6.6. I have rule defined as mentioned below destination d_networkx { file("/mnt/u001/syslog/$HOST/$YEAR$MONTH$DAY/network.log"); }; But syslog-ng is unable to write some logs to local file even though we see those logs in the tcpdump? What could be causing this. And one more thing need to know , is ther any way to check per filter (d_networkx) how many we received and how many we Processed(wrote) to a file Regards Harish Shetty

3 2

Insider 2018-09: SCL; Splunk .conf18; failover; human readable numbers; webinars;
by Czanik, Péter 20 Sep '18

20 Sep '18

Dear syslog-ng users, This is the 69th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Creating your first block for the syslog-ng configuration library (SCL) --------------------------------------------------------- The syslog-ng configuration library (SCL) is a collection of ready-to-use configuration snippets that hide away the complexity of the specifics of your log processing pipeline. If you already reuse parts of your configuration on different machines, it is not rocket science to turn these parts into blocks, making your configuration files a lot simpler. And if they are not site-specific, you can even help other syslog-ng users by submitting them to be included in SCL, and distributed in the official syslog-ng releases. https://www.syslog-ng.com/community/b/blog/posts/creating-your-first-block-… syslog-ng at Splunk .conf18 --------------------------- Going to Splunk .conf18? See a demo, meet with us (Booth #M49) and talk to syslog-ng creator, Balázs Scheidler. He will also give a talk on 02 October, 2:15 - 3:00 p.m. about simplifying complex log data ingestion with syslog-ng: https://conf.splunk.com/learn/session-catalog.html?search=ITS2133#/ Client-side failover and failback using syslog-ng ------------------------------------------------- When you have multiple syslog servers collecting logs, syslog-ng, on the client side, can fail over to secondary servers if the primary one becomes unavailable. It can also fail back to the primary server soon after it is back online – if configured so. Using client-side failover lowers the risk of message loss and makes the maintenance of servers easier. https://www.syslog-ng.com/community/b/blog/posts/client-side-failover-and-f… Human readable numbers in syslog-ng ----------------------------------- Writing an extra zero at the wrong place in a configuration file can result in a filled-up partition and data loss. This is where human readable numbers in syslog-ng can help: using kilo / mega / giga instead of writing many-digit numbers can save you lots of debugging and headache. https://www.syslog-ng.com/community/b/blog/posts/human-readable-numbers-in-… WEBINARS You can watch our past webinars: * Logging in docker with syslog-ng: https://www.brighttalk.com/webcast/16207/333091 * Supercharge your syslog-ng parsing: https://www.brighttalk.com/webcast/16207/330198 Or register for our upcoming syslog-ng webinars: * High performance log streaming to HDFS with syslog-ng: https://www.syslog-ng.com/event/high-performance-log-streaming-to-hdfs-with… Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> One Identity / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Mutual TLS issues
by Matthew Pounsett 19 Sep '18

19 Sep '18

I'm trying to get mutual TLS validation working and seem to be having issues with the config. The situation I've got is that I'm trying to accept a stream from another organization. And so my preferred configuration involves two CA's.. mine and theirs. My syslog-ng server needs to validate their server certificate using their CA certificate, and their syslog-ng client needs to validate my server certificate using my CA certificate. I've set up a test using the server that will be production, and a handy VM I control. I assumed that it would be as simple as using the example configs from the manual [0], and anywhere a CA cert is involved just pick the appropriate one: store their CA on my server, and my CA on "their" server (the test VM). That didn't work. I've backed down to trying to get this working using all certificates signed by my CA, but I can't seem to make that work either. The error timing I'm getting doesn't quite match similar errors I can see reported online. If I don't have the correct CA cert on the client, then I see a connect, immediate disconnect, and a "SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca" error on the server. If I do have the CA cert on the client, I see a connect ... and then nothing until I try to send a message. Then I get the same error on the server and a logged "Certificate validation failed" error on the client referencing the server's cert, and a disconnect. Server logs: Sep 17 23:17:49 adm syslog-ng[31128]: Syslog connection accepted; fd='24', client='AF_INET(216.235.10.40:54063)', local='AF_INET(64.191.0.17:6514)' Sep 17 23:18:01 adm syslog-ng[31128]: SSL error while reading stream; tls_error='SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca' Sep 17 23:18:01 adm syslog-ng[31128]: I/O error occurred while reading; fd='24', error='Connection reset by peer (104)' Sep 17 23:18:01 adm syslog-ng[31128]: Syslog connection closed; fd='24', client='AF_INET(216.235.10.40:54063)', local='AF_INET(64.191.0.17:6514)' Client logs: Sep 17 23:17:48 ubuntu syslog-ng[4603]: syslog-ng starting up; version='3.13.2' Sep 17 23:17:48 ubuntu syslog-ng[4603]: Syslog connection established; fd='13', server='AF_INET(64.191.0.17:6514)', local='AF_INET(0.0.0.0:0)' Sep 17 23:18:00 ubuntu syslog-ng[4603]: Certificate validation failed; subject='emailAddress=xxx(a)dns-oarc.net, CN=64.191.0.17, O=OARC\, Inc., L=Indianapolis, ST=Indiana, C=US', issuer='emailAddress=xxx(a)dns-oarc.net, CN=OARC Inc. Certificate Authority, O=OARC\, Inc., ST=Indiana, C=US', error='unable to get local issuer certificate', depth='0' Sep 17 23:18:00 ubuntu syslog-ng[4603]: SSL error while writing stream; tls_error='SSL routines:tls_process_server_certificate:certificate verify failed' Sep 17 23:18:00 ubuntu syslog-ng[4603]: I/O error occurred while writing; fd='13', error='Broken pipe (32)' Sep 17 23:18:00 ubuntu syslog-ng[4603]: Syslog connection broken; fd='13', server='AF_INET(64.191.0.17:6514)', time_reopen='60' Note the 12 second gap between startup and cert validation failure. The failure coincides with logging a message that should be sent from the client to the server. Server config: source tls_source { network(ip(64.191.0.17) port(6514) transport("tls") tls( key-file("/etc/ssl/private/64.191.0.17.key") cert-file("/etc/ssl/certs/64.191.0.17.crt") ca-dir("/etc/ssl/certs") ) ); }; Client config: destination d_tls { network("64.191.0.17" port(6514) transport("tls") tls( ca-dir("/etc/ssl/certs") key-file("/etc/ssl/private/216.235.10.40.key") cert-file("/etc/ssl/certs/216.235.10.40.crt") ) ); }; I have the CA cert stored as /etc/ssl/certs/server_ca.pem on both systems. Any idea what's wrong here? And, should I be able to get this to work using different CAs for the client and server certs? Thanks [0]: <https://www.syslog-ng.com/technical-documents/doc/ syslog-ng-open-source-edition/3.16/mutual-authentication- using-tls#TOPIC-956368>

2 3

@ as delimiter in patterndb parser
by joao reis 13 Sep '18

13 Sep '18

How can I specify @ as a delimiter in patterndb? I want to parse user and domain separated in email adresses. Thanks in advance.

2 1

Packet loss.
by vijay amruth 13 Sep '18

13 Sep '18

Hello everyone, we’ve been experiencing packet loss on syslog-ng collectors (these are open stack VMs), lately we have increased ingestion and that refelcted packet loss ( especially UDP). Tuning parameters didn’t help. If anyone here on the list has experienced a similar issue please share your resolution or work around. We are at 3.9.1, we are planning to update to 3.17.x for better data handling/other bug fixes. As an alternate approach we have increased the VM capacity ( no of VMs). If there is any that I missed, please point. Any thoughts? Thank you, Vijay Amrut. -- Thanks, Vijay Amrut.

3 4

Crash on syslog-ng 3.8.1
by xen rkar 12 Sep '18

12 Sep '18

Hi, Looking to see if this is a know issue, cause i have been unsucessful in find an exact match of the signature anywhere in the forums. Regards, rkar #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x76c5775c in __GI_abort () at abort.c:79 #2 0x76f537a8 in iv_fatal (fmt=0x76f750dc "iv_timer_unregister: timer index %d > %d") at ../../../../syslog-ng-3.8.1/lib/ivykis/src/iv_fatal.c:57 #3 0x76f53f4c in iv_timer_unregister (_t=0x17e5284) at ../../../../syslog-ng-3.8.1/lib/ivykis/src/iv_timer.c:305 #4 0x76f53fb8 in iv_run_timers (st=st@entry=0x17a50c0) at ../../../../syslog-ng-3.8.1/lib/ivykis/src/iv_timer.c:148 #5 0x76f55ad8 in iv_main () at ../../../../syslog-ng-3.8.1/lib/ivykis/src/iv_main_posix.c:107 #6 0x76f24524 in main_loop_run () at ../syslog-ng-3.8.1/lib/mainloop.c:497 #7 0x000113e0 in main (argc=70624, argv=0x7ea96054) at ../syslog-ng-3.8.1/syslog-ng/main.c:289

2 1