- syslog-ng - lists.balabit.hu

SYSLOG to SNMP TRAP relay server, how can I spoof source address
by Armando Martires 23 Jul '19

23 Jul '19

Hello, I set up a Syslog-ng relay server to relay syslog messages as SNMP traps. The relay is working sending a SNMP TRAP whenever a SYSLOG message is received, but the source seems to be the syslog-ng server and not the client host that originated the syslog message. This is my config: options { time_reopen (10); log_fifo_size (1000); chain_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); chain_hostnames (no); }; source s_labs_itsm { syslog(ip("192.168.50.20") transport("udp")); }; destination d_local { file("/var/log/messages_${HOST}"); }; destination d_ss { snmp( host("192.168.50.10") version("v2c") community("public") trap-obj('.1.3.6.1.6.3.1.1.4.1.0', 'Objectid', '.1.3.6.1.4.1.18372.3.1.1.1.2.1') snmp-obj('.1.3.6.1.4.1.18372.3.1.1.1.1.1.0', 'Octetstring', '${MESSAGE}') snmp-obj('.1.3.6.1.4.1.18372.3.1.1.1.1.2.0', 'Octetstring', 'admin') snmp-obj('.1.3.6.1.4.1.18372.3.1.1.1.1.3.0', 'Ipaddress', '${HOST}') ); }; log { source(s_labs_itsm); destination(d_local); destination(d_ss); }; Can anyone help me out understanding what I'm doing wrong? Thanks! -- Armando Mártires

2 1

Syslog-ng header & rfc5424
by Florian Goulais 19 Jul '19

19 Jul '19

Hi. I'am trying to understand why syslog-ng and an header (timestamp host) to log as the rfc specify another header. Someone can explain me the reason plz ? Regards Florian

3 6

Get local log files written immediately even if remote log server is unresponsive
by Jim Segrave 16 Jul '19

16 Jul '19

We are running syslog-ng 3.16.1 on Centos 7.4.1708 on a central logging host. We have a large (nearly 1,000) servers also running the same version of syslog-ng on the same Centos release. The servers are configured to log locally and also forward logs to the central logging host. This morning we encountered a problem - syslog-ng was running on the logging host, but was not processing incoming logs or locally generated ones. The servers forwarding to the central host did not write anything to their local log files, a small but significant portion of them had syslog crash, after which is was restarted by systemd, but still no logs were written until syslog-ng was forcibly stopped on the central server and then restarted. Connections to the central server weren't failing in the sense of TCP close or reset, but logs were accumulating on all the servers, including the central one, in the cache file for buffering logs. For our purposes, we need to have up-to-the moment logs available on the individual servers, so an admin going in to troubleshoot on a server who only has console access still has recent logs to consult if needed. Is there a way to tell syslog-ng to write local logs immediately even if it's currently buffering logs for sending to a non-responsive remote server?

3 3

Missing messages
by Klimek, Bryan J. 16 Jul '19

16 Jul '19

First time poster, so be gentle. We run syslog-ng in our environment as a centralized syslog manager for the archiving of syslog data. We have over 2000+ Unix/Linux systems sending their syslog data with a daily ingest rate of about 10GB per day. It was recently pointed out to me that one particular message from one specific host is not getting persisted to the files on our syslog-ng server all the time. That is to say, it is intermittent. If one person can find one message that is not making into our syslog-ng archive, I can only assume that we are dropping other messages as well. How can I debug if and when messages are being lost and not making into the files on my centralized syslog server? Bryan Klimek Mayo Clinic

4 10

Cannot send Syslog-ng to Elasticsearch
by Allen Olivas 12 Jul '19

12 Jul '19

Hello, Recently I've tried following along with the Syslog-NG to Elasticsearch and Kibana blog posts and Admin Documentation for integrating Syslog-NG into Elasticsearch but I'm unable to integrate the two. I see in the .conf files the destination calls for creating and Index Pattern for Syslog-NG but when I curl the existing indices I do not see syslog-ng. Also, I'm now receiving two errors. The first I'm fairly certain we need to resolve but I've not been able to find adequate documentation on how to identify the issue let along resolve it, and the second I'm not sure if we actually need to fix. The two issues: Issue#1: I/O error occurred syslog-ng[26432]: Syslog connection established; fd='12', server='AF_INET(127.0.0.1:9200)', local='AF_INET(0.0.0.0:0)' syslog-ng[26432]: I/O error occurred while writing; fd='12', error='Broken pipe (32)' syslog-ng[26432]: Syslog connection broken; fd='12', server='AF_INET(127.0.0.1:9200)', time_reopen='60' Issue#2: Error opening plugin module; module='mod-java', error='libjvm.so: cannot open shared object file: No such file or directory' For issue 1 I'm not sure what to do or how to resolve it. For issue 2, I know for certain libjvm does exist, and I've mapped the LD_LIBRARY_PATH to the directory libjvm.so resides in. Ultimately, are these two issues preventing Syslog-NG from sending to Elasticsearch or are they just separate issues to tackle after I get things cleared up, and most importantly if they're not related, how do I integrate Syslog-NG with Elasticsearch and Kibana. Documentation is not helpful and not concise. Thanks!

4 18

Periodic "Destination timed out, reaping" and disconnection on a write fifo (pipe)
by Charles Eidsness 27 Jun '19

27 Jun '19

I kind of feel like this is going to be a dumb question but I just can't figure this out. I'm trying to get syslog-ng working with sagan, but every 60 seconds syslog-ng closes the fifo it's writing to, which closes the sagan connection to the fifo. When running in verbose mode syslog-ng is throwing up a "Destination timed out, reaping..." message. I simplified my syslog-ng configuration to: source s_sys { system(); }; destination d_fifo { pipe( "/tmp/test.fifo" ); }; log { source(s_sys); destination(d_fifo); }; And then run use cat to read the fifo: cat /tmp/test.fifo It sends all of the log traffic every but every 60 seconds cat closes. Is this expected behavior? Am I missing something? I'm using syslog-ng version 3.19.1. Thanks! Charles == Charles Eidsness Chief Engineer *CCX Technologies* (613)791-9715

2 7

Sending Messages to Slack via a HTTP Proxy
by Frank Crawford 26 Jun '19

26 Jun '19

We have recently upgraded to syslog-ng v3.19 so we can use the slack hook, but have an issue that to access the URL we will need to go through a squid web proxy. How do we configure the details of the proxy in either syslog-ng or in the slack destination? Regards Frank -- [http://www.ac3.com.au/emailsigs/v2/ac3_logo.png]<http://www.ac3.com.au> the human touch in a technical world Frank Crawford Technology and Infrastructure Manager t 02 9199 0995 | m 0401 714 870 e frank.crawford(a)ac3.com.au<mailto:frank.crawford@ac3.com.au> Level 8 East, 8 Central Ave, Eveleigh NSW 2015 [http://www.ac3.com.au/emailsigs/v2/linkedin.png] <https://www.linkedin.com/company/ac3> [http://www.ac3.com.au/emailsigs/v2/twitter.png]<https://twitter.com/ac3ptyltd> [http://www.ac3.com.au/emailsigs/v2/iso.png]<http://www.ac3.com.au/about-us/news/double-iso-certification-ac3> This email is for intended recipients only. If you are not an intended recipient, you must not disclose, copy or distribute this email to others or act upon the information contained in this email in any way. This email may contain privileged or confidential information. If you have received this email in error, please notify the sender and then delete it from your system.

2 4

error on syslog-ng elasticsearch2 destination
by joao reis 26 Jun '19

26 Jun '19

Hi, I'm getting the following error when trying to configure the elasticseach2 backend: Error parsing destination statement, destination plugin java not found in block destination elasticsearch2() at /usr/share/syslog-ng/include/scl/elasticsearch/plugin.conf:25:4:3-4:7: 1 2 #Start Block block destination elasticsearch2() at /usr/share/syslog-ng/include/scl/elasticsearch/plugin.conf:25 3 4-----> java( 4-----> ^^^^ 5 class_path("//usr/lib64/syslog-ng/java-modules/*.jar://usr/lib64/syslog-ng/java-modules/elastic-jest-client/*.jar:/*.jar") 6 class_name("org.syslog_ng.elasticsearch_v2.ElasticSearchDestination") 7 option("index", "logserver1") 8 option("type", "syslog") 9 option("server", "localhost") This is the block that causes the error: destination d_es { elasticsearch2 ( cluster("logserver1") client-mode("http") index("logserver1") type("syslog") template("$(format-json --scope rfc5424 --scope nv-pairs --exclude DATE --key ISODATE)") ); }; root@elk-server1 conf.d]# rpm -qa|grep syslog-ng syslog-ng-3.21.1-3.el7.x86_64 syslog-ng-java-3.21.1-3.el7.x86_64 syslog-ng-http-3.21.1-3.el7.x86_64 Any ideia? Thanks in advance.

1 1

syslog-ng-3.22: unofficial Debian/Ubuntu packages
by Laszlo Budai 25 Jun '19

25 Jun '19

Hi, syslog-ng 3.22.1[1] packages are available in OBS repo[2]. List of supported OSs: * Debian 8.0 * Debian 9.0 [including armv7l] * Raspbian 9.0 * Ubuntu 14.04 * Ubuntu 16.04 * Ubuntu 18.04 * Ubuntu 18.10 Install ------- example: Debian 9.0 1. get release key wget -qO - http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… | sudo apt-key add - 2. add repo to APT sources eg.: /etc/apt/sources.list.d/syslog-ng-obs.list deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… ./ Then `apt-get update` and `apt-get install syslog-ng-core` Links -------- [1] https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.22.1 [2] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… regards, Laszlo Budai

1 0

syslog-ng-3.22.1
by Laszlo Budai 25 Jun '19

25 Jun '19

3.22.1 Highlights * Sending SNMP traps: Using the new snmp() destination, incoming log messages can be converted to SNMP traps, as the fields of the SNMP messages can be customized with macros. (#2693<https://github.com/balabit/syslog-ng/pull/2693>) * $(template) dynamic binding: Extends the $(template) template function to allow dynamic binding. For example, the name of the template to be invoked can come from the message (name-value pairs). (#2716<https://github.com/balabit/syslog-ng/pull/2716>) * syslog(), network(): Add dynamic-window-size() option to enable dynamic flow control that distributes the specified amount of window between active connections at runtime. This can be used in low-memory environments, where only a small subset of the active clients sends messages at high rate. (#2772<https://github.com/balabit/syslog-ng/pull/2772>) Features * match(): Add support for the template() option (#2715<https://github.com/balabit/syslog-ng/pull/2715>) * add-contextual-data(): Allow using templates in name-value pairs (#2711<https://github.com/balabit/syslog-ng/pull/2711>) * Add support for floating point operations in template functions (#2742<https://github.com/balabit/syslog-ng/pull/2742>) * Add support for usec precision when parsing time (#2709<https://github.com/balabit/syslog-ng/pull/2709>) Bugfixes * Fix null pointer access when destinations are suspended (#2778<https://github.com/balabit/syslog-ng/pull/2778>) * Fix grouping-by() deadlock (#2758<https://github.com/balabit/syslog-ng/pull/2758>) * Fix a general source-related crash and enhance wildcard-file()'s bookmark handling (#2589<https://github.com/balabit/syslog-ng/pull/2589>) * Fix infinite loop (reload/reopen) (#2739<https://github.com/balabit/syslog-ng/pull/2739>) * Fix python() package/module name collision (#2438<https://github.com/balabit/syslog-ng/pull/2438>) * Fix escaped quote in block argument (#2781<https://github.com/balabit/syslog-ng/pull/2781>) * Reintroduce test on SYSLOG_NG_HAVE_TIMEZONE (#2774<https://github.com/balabit/syslog-ng/pull/2774>) * snmp(): Fix template leak (#2746<https://github.com/balabit/syslog-ng/pull/2746>) Other changes * Never drop flow-controlled messages: The meaning of log-fifo-size() has changed to avoid dropping flow-controlled messages when log-fifo-size() is misconfigured. From now on, log-fifo-size() only affects messages that are not flow-controlled. (#2753<https://github.com/balabit/syslog-ng/pull/2753>) * The -d/--debug syslog-ng command line flag no longer implies -e/--stderr. If you want to redirect internal() source to stderr, use the -e/--stderr option explicitly. (#2731<https://github.com/balabit/syslog-ng/pull/2731>) * dbld, RPM and DEB packaging improvements (#2724<https://github.com/balabit/syslog-ng/pull/2724>) * Checkpoint parser improvements (#2740<https://github.com/balabit/syslog-ng/pull/2740>) * Reset the timezone on config reload event (#2691<https://github.com/balabit/syslog-ng/pull/2691>) * geoip2(): Include IP into the error message (#2743<https://github.com/balabit/syslog-ng/pull/2743>) * Improve regexp error messages (#2796<https://github.com/balabit/syslog-ng/pull/2796>) * http(): Warn if less workers used than urls (#2757<https://github.com/balabit/syslog-ng/pull/2757>) * http(): Allow URLs to be specified by a space/comma separated string (#2699<https://github.com/balabit/syslog-ng/pull/2699>) * loggen: Change message rate at runtime using signals (#2756<https://github.com/balabit/syslog-ng/pull/2756>) * debun: add acquire_running_syslog_config function (#2752<https://github.com/balabit/syslog-ng/pull/2752>) * FreeBSD fixes for the test suite (#2783<https://github.com/balabit/syslog-ng/pull/2783>) Notes to the developers * ivykis: update to 0.42.4 (#2736<https://github.com/balabit/syslog-ng/pull/2736>) * Support generator plugins in global options (#2747<https://github.com/balabit/syslog-ng/pull/2747>) * logthrfetcher: new constants (#2766<https://github.com/balabit/syslog-ng/pull/2766>) * logthrsourcedrv: support position tracking (#2750<https://github.com/balabit/syslog-ng/pull/2750>) * Light: Support pre-commit and tox (#2725<https://github.com/balabit/syslog-ng/pull/2725>) * Enable Bison error flags: conflicts-sr/rr (#2762<https://github.com/balabit/syslog-ng/pull/2762>) * Dynamic stats constant registration (#2761<https://github.com/balabit/syslog-ng/pull/2761>) Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Andras Mitzki, Antal Nemes, Attila Szakacs, Balazs Scheidler, Christian Michallek, Fabien Wernli, Gabor Nagy, Kyeong Yoo, Laszlo Budai, Laszlo Szemere, László Várady, Mehul Prajapati, Norbert Takacs, Oleksii Hamov, Péter Kókai, Romain Tartière, Zoltan Pallagi. https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.22.1 [https://avatars1.githubusercontent.com/u/986981?s=400&v=4]<https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.22.1> balabit/syslog-ng<https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.22.1> syslog-ng is an enhanced log daemon, supporting a wide range of input and output methods: syslog, unstructured text, queueing, SQL & NoSQL. - balabit/syslog-ng github.com

1 0