July 2019 - syslog-ng - lists.balabit.hu

cisco templetes
by ciprian niculescu 31 Jul '19

31 Jul '19

Hello, i'm building a syslog relay to collect and duplicate the flows to multiple destinations. but the relayed messages are strange looking. my source are cisco network devices (catalyst, nexus, asa) and i want to relay to a Solarwinds, Splunk and a linux-syslog for archiving. i search the net for a templete but found none. What i got so far is that the catalyst is sending in syslog bsd format, but with the relay configured to source bsd and destination bsd, the end message is different (the date is doubled, the relay add his IP) any help is appreciated. Regards, Ciprian

2 1

Tips for handling large message load
by Faine, Mark R. (MSFC-IS40)[NICS] 30 Jul '19

30 Jul '19

I have several Splunk log aggregators that gets thousands of messages per second but we are seeing issues with dropping messages from UDP sources. I've read the section in the docs about handling large message load and we've made many of those changes. Do you have any other suggestions to improve performance? We are using flow control. We have made the following sysctl changes: - net.core.rmem_max = 268435456 - net.core.netdev_max_backlog = 2000 We have increased flush-lines to 100 We are also looking to increase the initial window size below and max connections as well as the so_rcvbuf. network(port(514) transport("tcp") max-connections(100) log_iw_size(10000) flags(syslog-protocol)); network(port(514) transport("udp") max-connections(100) log_iw_size(10000) flags(syslog-protocol)); network(ip(0.0.0.0) transport("tls") port(10514) max-connections(100) log_iw_size(10000) flags(syslog-protocol) I'm currently showing about 50 TCP connections and almost 1000 UDP connections, though it's early on Monday morning so it is likely to increase as the day goes on. Most of the log messages are coming from the VPNs (hence so many UDP connections). The servers are quite beefy with 64GB of RAM and 24 Xeon cores @ 2.4GHz (Dell PowerEdge R530). We are using two separate network interfaces one for TCP and the other for UDP. I'd appreciate any suggestions on how to further increase performance, also, any general rules or calculations I can use to determine optimal values for these parameters from available system metrics would be very helpful. Thanks, -Mark

3 2

SYSLOG to SNMP TRAP relay server, how can I spoof source address
by Armando Martires 23 Jul '19

23 Jul '19

Hello, I set up a Syslog-ng relay server to relay syslog messages as SNMP traps. The relay is working sending a SNMP TRAP whenever a SYSLOG message is received, but the source seems to be the syslog-ng server and not the client host that originated the syslog message. This is my config: options { time_reopen (10); log_fifo_size (1000); chain_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); chain_hostnames (no); }; source s_labs_itsm { syslog(ip("192.168.50.20") transport("udp")); }; destination d_local { file("/var/log/messages_${HOST}"); }; destination d_ss { snmp( host("192.168.50.10") version("v2c") community("public") trap-obj('.1.3.6.1.6.3.1.1.4.1.0', 'Objectid', '.1.3.6.1.4.1.18372.3.1.1.1.2.1') snmp-obj('.1.3.6.1.4.1.18372.3.1.1.1.1.1.0', 'Octetstring', '${MESSAGE}') snmp-obj('.1.3.6.1.4.1.18372.3.1.1.1.1.2.0', 'Octetstring', 'admin') snmp-obj('.1.3.6.1.4.1.18372.3.1.1.1.1.3.0', 'Ipaddress', '${HOST}') ); }; log { source(s_labs_itsm); destination(d_local); destination(d_ss); }; Can anyone help me out understanding what I'm doing wrong? Thanks! -- Armando Mártires

2 1

Syslog-ng header & rfc5424
by Florian Goulais 19 Jul '19

19 Jul '19

Hi. I'am trying to understand why syslog-ng and an header (timestamp host) to log as the rfc specify another header. Someone can explain me the reason plz ? Regards Florian

3 6

Get local log files written immediately even if remote log server is unresponsive
by Jim Segrave 16 Jul '19

16 Jul '19

We are running syslog-ng 3.16.1 on Centos 7.4.1708 on a central logging host. We have a large (nearly 1,000) servers also running the same version of syslog-ng on the same Centos release. The servers are configured to log locally and also forward logs to the central logging host. This morning we encountered a problem - syslog-ng was running on the logging host, but was not processing incoming logs or locally generated ones. The servers forwarding to the central host did not write anything to their local log files, a small but significant portion of them had syslog crash, after which is was restarted by systemd, but still no logs were written until syslog-ng was forcibly stopped on the central server and then restarted. Connections to the central server weren't failing in the sense of TCP close or reset, but logs were accumulating on all the servers, including the central one, in the cache file for buffering logs. For our purposes, we need to have up-to-the moment logs available on the individual servers, so an admin going in to troubleshoot on a server who only has console access still has recent logs to consult if needed. Is there a way to tell syslog-ng to write local logs immediately even if it's currently buffering logs for sending to a non-responsive remote server?

3 3

Missing messages
by Klimek, Bryan J. 16 Jul '19

16 Jul '19

First time poster, so be gentle. We run syslog-ng in our environment as a centralized syslog manager for the archiving of syslog data. We have over 2000+ Unix/Linux systems sending their syslog data with a daily ingest rate of about 10GB per day. It was recently pointed out to me that one particular message from one specific host is not getting persisted to the files on our syslog-ng server all the time. That is to say, it is intermittent. If one person can find one message that is not making into our syslog-ng archive, I can only assume that we are dropping other messages as well. How can I debug if and when messages are being lost and not making into the files on my centralized syslog server? Bryan Klimek Mayo Clinic

4 10

Cannot send Syslog-ng to Elasticsearch
by Allen Olivas 12 Jul '19

12 Jul '19

Hello, Recently I've tried following along with the Syslog-NG to Elasticsearch and Kibana blog posts and Admin Documentation for integrating Syslog-NG into Elasticsearch but I'm unable to integrate the two. I see in the .conf files the destination calls for creating and Index Pattern for Syslog-NG but when I curl the existing indices I do not see syslog-ng. Also, I'm now receiving two errors. The first I'm fairly certain we need to resolve but I've not been able to find adequate documentation on how to identify the issue let along resolve it, and the second I'm not sure if we actually need to fix. The two issues: Issue#1: I/O error occurred syslog-ng[26432]: Syslog connection established; fd='12', server='AF_INET(127.0.0.1:9200)', local='AF_INET(0.0.0.0:0)' syslog-ng[26432]: I/O error occurred while writing; fd='12', error='Broken pipe (32)' syslog-ng[26432]: Syslog connection broken; fd='12', server='AF_INET(127.0.0.1:9200)', time_reopen='60' Issue#2: Error opening plugin module; module='mod-java', error='libjvm.so: cannot open shared object file: No such file or directory' For issue 1 I'm not sure what to do or how to resolve it. For issue 2, I know for certain libjvm does exist, and I've mapped the LD_LIBRARY_PATH to the directory libjvm.so resides in. Ultimately, are these two issues preventing Syslog-NG from sending to Elasticsearch or are they just separate issues to tackle after I get things cleared up, and most importantly if they're not related, how do I integrate Syslog-NG with Elasticsearch and Kibana. Documentation is not helpful and not concise. Thanks!

4 18