- syslog-ng - lists.balabit.hu

syslog-ng: unofficial 3.23 Debian/Ubuntu packages
by Laszlo Budai 26 Aug '19

26 Aug '19

Hi, syslog-ng 3.23.1[1] packages are available in OBS repo[2]. List of supported OSs: * Debian 8.0 * Debian 9.0 [including armv7l] * Raspbian 9.0 * Ubuntu 14.04 * Ubuntu 16.04 * Ubuntu 18.04 * Ubuntu 18.10 Install ------- example: Debian 9.0 1. get release key wget -qO - http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… | sudo apt-key add - 2. add repo to APT sources eg.: /etc/apt/sources.list.d/syslog-ng-obs.list deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… ./ Then `apt-get update` and `apt-get install syslog-ng-core` Links -------- [1] https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.23.1 [2] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… regards, Laszlo Budai [Twitter]<https://twitter.com/Budai_L> [LinkedIn] <https://www.linkedin.com/in/laszlo-budai-55243a3/> [Github] <https://github.com/lbudai>

1 0

syslog-ng-3.23
by Laszlo Budai 26 Aug '19

26 Aug '19

3.23.1 ====== ## Features * The `redis()` destination now handles any number of command parameters. (#2816) * The `format()` option of `date-parser()` supports lists. From now on, a single `date-parser()` instance is able to process different date formats, making it easy to catch on when some programs change the way they log information. (#2779) * Add relocation support for `disk-buffer()`. The `relocate` subcommand of `dqtool` can be used to move a single or multiple queue files. (#2855) * `file(), pipe()`: The `time-reap()` option now can be set or disabled for each destination separately. Use `time-reap(0)` to disable closing idle destination files. (#2798) * `syslog-ng-ctl`: Add `list-files` subcommand to print files present in the current configuration. (#2797) ## Bugfixes * Fix minor memory leaks (#2868) * Add global context to standalone parsers (#2876) * Fix heap usage after free in cfg_run_parser_with_main_context (#2884) * Fix g_thread_init call order issue with older glibs (#2853) * SNMP destination: fixing statistics format (#2854) * eventlog: flush escaped_buffer when full (#2837) * dbparser: remove unnecessary lock (#2838) * dbparser: fix crash when context times out in the middle of another rule (#2832) * radix: fix grouping in PCRE (#2808) * add-contextual-data: make filters config plugin aware (#2886) * Undefined warning regression (#2829) ## Other changes * Disable `time-reap()` on non-templated filenames by default (#2798) * The `--preprocess-into` command line flag accepts `-`, and writes the preprocessed configuration to stdout. (#2869) * Add information on environment variables passed to the confgen script (#2888) * Light: minor fixes (#2867, #2844) * python: use malloc_debug for python unit tests (#2866) * Travis: verbose unit test output (#2851) * Travis cleanup (#2809) * filter: add unit tests to `filter-op` (#2835) * Fix clang/gcc diagnostic differences (#2810) * Fix leak in stats test (#2874) * cmake, autotools: -Wundef to enable-extra-warning (#2806) ## Notes to the developers * Check whether commit messages are properly formatted (#2803, #2807) * gitignore: tags file and build directory (#2794) ## Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Andras Mitzki, Antal Nemes, Attila Szakacs, Balazs Scheidler, Gabor Nagy, Laszlo Budai, Laszlo Szemere, László Várady, Mark Bonsack, Mehul Prajapati, Péter Kókai, Romain Tartière, Zoltan Pallagi. https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.23.1

1 0

IP WHITE LIST
by Onur GURSOY 21 Aug '19

21 Aug '19

Hello Guys, Nowadays, im finding whitelist method for IP. Is it possible ? How can i allow IP which is in whitelist ? So Many Thanks, My Best Regards. -- Onur GÜRSOY R&D Engineer in Embedded Systems Master Student at Gebze Institute Of Technology Department Of Electronic Engineering GSM : 0(545) 764 7653 e-mail: onurgursoygyte(a)gmail.com

2 1

Dynamic collection name in MongoDb
by PC LP 15 Aug '19

15 Aug '19

Hi, I am retrieving the value for *$M_ADDRESS* from the Syslog message using patterndb. $M_ADDRESS is a unique value. Based on that I have to create the collection in MongoDB dynamically. If I provide the value like below it is not recognizing collection("$M_ADDRESS") Is there any alternate way to create it? Please advise. Regards, PCLP.

2 1

if condition
by Alexey Vlasov 15 Aug '19

15 Aug '19

Hi, I would like to optimize my syslong-ng.conf file thus it could be used on some boxes without any changing. Now I have a few modifications of syslong-ng.conf on each server to copy logs to remote box. For example, group of hosts A* ============================== A1-machine (it is its hostname) has: destination clnt { udp("remote-box-a" port(514)); }; log { source(src); filter(f_messages); destination(clnt); }; ============================== ============================== A2-machine has: destination clnt { udp("remote-box-a" port(514)); }; log { source(src); filter(f_messages); destination(clnt); }; ============================== and group of hosts B* ============================== B1-machine has: destination clnt { udp("remote-box-b" port(514)); }; log { source(src); filter(f_messages); destination(clnt); }; ============================== ============================== B2-machine has: destination clnt { udp("remote-box-b" port(514)); }; log { source(src); filter(f_messages); destination(clnt); }; ============================== Is there possible to write something like that If (${HOSTNAME} ~ ^A) use remote-box-a If (${HOSTNAME} ~ ^B) use remote-box-b Thanks in advance!

2 1

3.22.1: understanding the location of the python's part
by Sergey 14 Aug '19

14 Aug '19

Hello I see that in 3.22 has changed the location of the python's part (relative 3.20): architecture independent files moved to architecture specific directory (lib64 on x86_64). For example, rhel spec 3.20 (syslog-ng-3.20.1/packaging/rhel/syslog-ng.spec) %files python /usr/lib/python2.7/site-packages/syslogng-1.0-py2.7.egg-info %dir /usr/lib/python2.7/site-packages/syslogng/ %dir /usr/lib/python2.7/site-packages/syslogng/debuggercli/ /usr/lib/python2.7/site-packages/syslogng/* /usr/lib/python2.7/site-packages/syslogng/debuggercli/* %{_libdir}/%{name}/libmod-python.so The same in 3.22 %files python %{_libdir}/%{name}/python/syslogng-1.0-py%{py_ver}.egg-info %{_libdir}/%{name}/python/syslogng/* %{_libdir}/%{name}/libmod-python.so Is the architecture specific file libmod-python.so only? Or am I missing something? -- Regards, Sergey

2 1

Parse message fields for use as columns in MySQL
by Allen Olivas 09 Aug '19

09 Aug '19

Hello, I'm really scratching my head trying to make this work and thought maybe the community has experienced this before. I'm collecting logs from Wazuh and Syslog-NG. Those logs are sent from my Wazuh server with Sylog-NG configured to send to my MySQL server. The Syslog-ng.conf file on the MySQL server is configured with a destination to mysql. In the declared the destination and list out columns and values. # MySQL define destination destination d_mysql { sql( type(mysql) username("syslog") password("xxxxxxx") database("syslog") host("127.0.0.1") table("logs") columns("host", "id", "location", "facility", "rule", "priority", "level", "tag", "datetime", "program", "msg") values("$HOST", "$ID", "$LOCATION","$FACILITY", "$RULE", "$PRIORITY", "$LEVEL", "$TAG","$YEAR-$MONTH-$DAY $HOUR:$MIN:$SEC","$PROGRAM", "$MSG") indexes("datetime", "host", "id", "location", "rule") ); }; So here's the problem. The Message data contains information like Rule and Location that really equate to the Wazuh Rule and Location = the Wazuh Agent that's reporting it. I had hoped "location" column would populate with the Location date from the message. Same with Rule. SO my question to the community is how on earth do I parse the data in the message field to populate columns (existing or new)? Any thoughts, guidance, recommendations are greatly appreciated. Thanks, Allen Olivas InfoDefense Office: (972) 848-7910 Email: allen.olivas(a)infodefense.com Toll Free: (877) INFODEFENSE www.infodefense.com<https://l.shatrk.com/r/e/DblvLSPvKY2IxMPE?r=https://app.salesforceiq.com/r?…>

2 5

Syslog-ng stops writing until restarted
by Faine, Mark R. (MSFC-IS40)[NICS] 07 Aug '19

07 Aug '19

I have a strange problem where logs are definitely coming in over tcp and udp however they are not written, well not until I restart the server. Whenever I restart the logs are written but shortly after starting they stop until the server is restarted again. Any ideas on where I can look? Thanks, -Mark

2 2

syslog-ng configuration from CLI
by SZIGETVÁRI János 06 Aug '19

06 Aug '19

Hello Team, A few years back I heard about a project that would have added functionality to syslog-ng that would have enabled configurability from the command line. This was the respective PR: https://github.com/balabit/syslog-ng/pull/1326 Since then the PR was closed and part of the functionality provided by the PR has been added to syslog-ng (in https://github.com/balabit/syslog-ng/pull/1605) but not the CLI configurability part. I personally think that being able to configure syslog-ng through its command line arguments would bear a great potential. Of course running it with a complex configuration passed in through CLI argument would be infeasible, but with simple, single-purpose logging setups it would prove to be a very powerful tool. For example when trying to empty a bunch of orphaned disk queue files, it would be very easy to spin up several syslog-ng instances from a script with only slightly differing configuration. What do you guys think? Do you agree as well that this feature would bear a lot of added value to syslog-ng? Thanks for sharing your opinions! Best Regards, János Szigetvári -- Janos SZIGETVARI RHCE, License no. 150-053-692 <https://www.redhat.com/rhtapps/verify/?certId=150-053-692> LinkedIn: linkedin.com/in/janosszigetvari __@__˚V˚ Make the switch to open (source) applications, protocols, formats now: - windows -> Linux, iexplore -> Firefox, msoffice -> LibreOffice - msn -> jabber protocol (Pidgin, Google Talk) - mp3 -> ogg, wmv -> ogg, jpg -> png, doc/xls/ppt -> odt/ods/odp

3 3

pattern documentation for patterndb
by Russell Fulton 06 Aug '19

06 Aug '19

I am sure I have asked this question before but I can’t find the answer now. The Admin Guide has a very sort section on patterns with a couple of examples. I can’t find the detailed docs for the actual pattern elements (parser types). I seem to remember that these we documented in a separate pdf but I can’t find it. I add new rules about once a year and struggle with it every time! Russell

2 1

Quoting in filters
by Faine, Mark R. (MSFC-IS40)[NICS] 05 Aug '19

05 Aug '19

What is the policy on quoting values in filters? In the docs it seems that sometimes they are quoted and other times they are not. For example, are these equivalent: netmask (127.0.0.1/32) and netmask("127.0.0.1/32") I think that maybe some filters, the ones that take strings essentially, are quoted, and not the ones that take numeric data. Is that correct? Thanks, -Mark

2 1

new install of OSE 3.22 fails on ubuntu 16,04
by Russell Fulton 05 Aug '19

05 Aug '19

Running on ubuntu 16.04 I have just compiled and installed syslog-ng version 3.22. When started it prints: iv_tls_user_register: called after iv_init and then exits. Google tells me there was a bug that caused this back in 2013 but there are not recent posts. Any idea what is going on? BTW I can’t use the binary packages as there are conflicts between the rsyslog and syslog-ng packages and I am stuck with having the rsyslog package by corporate policy. Russell

2 1

Fwd: Multithreaded inputs with same output file : is it safe ?
by Jean-Baptiste Fuzier 02 Aug '19

02 Aug '19

Hello, I have a few questions related to multithreading. >From the doc, my understanding is that inputs are multithreaded this way : 1/ Tcp connections are sent to different thread when multithreading is enabled, even if there is a single source enabled in config with a single tcp port listening 2/ UDP datagrams are NOT sent to different thread if there is a single source even if there are multiple listening ports in the source config 3/ UDP datagrams are sent to different threads if there are different sources in config (1 source -> 1 thread), so basically to benefit from multithreading with UDP one needs to configure several sources. Is my understanding correct ? If we have several UDP source with some round robin balancing of the input to different source on the same daemon process, that means that several host with the same name can send their syslog data to different source. For the output, I am using files with the name of the sending host in the name. Is there an issue if several source (= from different threads) ends up being outputed in the same file. Another way to turn the question is: are all the data from the source thread then send to some common thread so that output to a file is always handled by the same thread even if log is received from different source threads ? I tried to ask the same question with a schematic : Case1 : +-------------------------+ +-------------------+ +-----------------------------+ +-----------+ +----------+ | | | | | | | host foo +------->+ |X----->+ thread1 some foo log +----->+ +--->+ thread handling foo.log +------>foo.log +-----------+ | iptables |X XX| some bar log | | black box magic | | | | round | XXXX +-------------------------+ | | +-----------------------------+ | robin | XXXXX | | | |XX XX+-------------------------+ | | +-----------------------------+ +-----------+ | X------>+ | | | | | | host bar +------->+ | | thread2 some foo log +----->+ +--->+ thread handling bar.log +------>bar.log +-----------+ +----------+ | some bar log | | | | | +-------------------------+ +-------------------+ +-----------------------------+ Case2 : +-------------------------+ +-----------+ +----------+ | | | host foo +------->+ |X----->+ thread1 some foo log +---------------------> +--------+ +-----------+ | iptables |X XX| some bar log +----------| |foo.log | | round | XXXX +-------------------------+ +------------> +--------+ 2 different thread writting | robin | XXXXX | | to same file | |XX XX+-------------------------+ | +---------->----------+ likely bad ! +-----------+ | X------>+ +--------+ |bar.log | | host bar +------->+ | | thread2 some foo log +--------------------->----------+ +-----------+ +----------+ | some bar log | +-------------------------+ One example config : # Notice that host name is used in output file name # There is no afinity between 1 host <> 1 udp port => logs from host foo are send to every source/udp port in a round robin fashion source s_syslog__udp5141{ network(ip(192.168.1.1) transport("udp") port(5141) flags("threaded")); }; source s_syslog__udp5142{ network(ip(192.168.1.1) transport("udp") port(5142) flags("threaded")); }; source s_syslog__udp5143{ network(ip(192.168.1.1) transport("udp") port(5143) flags("threaded")); }; source s_syslog__udp5144{ network(ip(192.168.1.1) transport("udp") port(5144) flags("threaded")); }; destination d_network_fw_misc { file("`syslog_log_root_path`/fw/fw/$FULLHOST/`naming_scheme`" group("logs_fw") dir-group("logs_fw") log-fifo-size(500000)); }; destination d_network_wifi { file("`syslog_log_root_path`/wifi/$FULLHOST/`naming_scheme`" group("logs_wifi") dir-group("logs_wifi") log-fifo-size(500000)); }; destination d_network_debug { file("`syslog_log_root_path`/debug/`naming_scheme`" group("logs_network_debug") dir-group("logs_network_debug") log-fifo-size(600000)); }; log { source(s_syslog__udp5141); source(s_syslog__udp5142); source(s_syslog__udp5143); source(s_syslog__udp5144); # Every channel of the junction received a copy of the message, unless there is a final flag before # Final flags are applied only within the junction and not globally junction { channel {filter(f_debug); destination(d_network_debug);}; channel {filter(f_network_wifi); destination(d_network_wifi); flags(final);}; channel {destination(d_network_fw_misc); flags(final);}; }; }; #Round robin with iptables : iptables -t nat -A PREROUTING -d 192.168.1.1 -i bond0 -p udp -m udp --dport 514 -m statistic --mode nth --every 4 --packet 0 -j REDIRECT --to-port 5141 iptables -t nat -A PREROUTING -d 192.168.1.1 -i bond0 -p udp -m udp --dport 514 -m statistic --mode nth --every 3 --packet 0 -j REDIRECT --to-port 5142 iptables -t nat -A PREROUTING -d 192.168.1.1 -i bond0 -p udp -m udp --dport 514 -m statistic --mode nth --every 2 --packet 0 -j REDIRECT --to-port 5143 iptables -t nat -A PREROUTING -d 192.168.1.1 -i bond0 -p udp -m udp --dport 514 -m statistic --mode nth --every 1 --packet 0 -j REDIRECT --to-port 5144 Thanks for your help -- Jean-Baptiste Fuzier

2 1

cisco templetes
by ciprian niculescu 31 Jul '19

31 Jul '19

Hello, i'm building a syslog relay to collect and duplicate the flows to multiple destinations. but the relayed messages are strange looking. my source are cisco network devices (catalyst, nexus, asa) and i want to relay to a Solarwinds, Splunk and a linux-syslog for archiving. i search the net for a templete but found none. What i got so far is that the catalyst is sending in syslog bsd format, but with the relay configured to source bsd and destination bsd, the end message is different (the date is doubled, the relay add his IP) any help is appreciated. Regards, Ciprian

2 1

Tips for handling large message load
by Faine, Mark R. (MSFC-IS40)[NICS] 30 Jul '19

30 Jul '19

I have several Splunk log aggregators that gets thousands of messages per second but we are seeing issues with dropping messages from UDP sources. I've read the section in the docs about handling large message load and we've made many of those changes. Do you have any other suggestions to improve performance? We are using flow control. We have made the following sysctl changes: - net.core.rmem_max = 268435456 - net.core.netdev_max_backlog = 2000 We have increased flush-lines to 100 We are also looking to increase the initial window size below and max connections as well as the so_rcvbuf. network(port(514) transport("tcp") max-connections(100) log_iw_size(10000) flags(syslog-protocol)); network(port(514) transport("udp") max-connections(100) log_iw_size(10000) flags(syslog-protocol)); network(ip(0.0.0.0) transport("tls") port(10514) max-connections(100) log_iw_size(10000) flags(syslog-protocol) I'm currently showing about 50 TCP connections and almost 1000 UDP connections, though it's early on Monday morning so it is likely to increase as the day goes on. Most of the log messages are coming from the VPNs (hence so many UDP connections). The servers are quite beefy with 64GB of RAM and 24 Xeon cores @ 2.4GHz (Dell PowerEdge R530). We are using two separate network interfaces one for TCP and the other for UDP. I'd appreciate any suggestions on how to further increase performance, also, any general rules or calculations I can use to determine optimal values for these parameters from available system metrics would be very helpful. Thanks, -Mark

3 2

SYSLOG to SNMP TRAP relay server, how can I spoof source address
by Armando Martires 23 Jul '19

23 Jul '19

Hello, I set up a Syslog-ng relay server to relay syslog messages as SNMP traps. The relay is working sending a SNMP TRAP whenever a SYSLOG message is received, but the source seems to be the syslog-ng server and not the client host that originated the syslog message. This is my config: options { time_reopen (10); log_fifo_size (1000); chain_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); chain_hostnames (no); }; source s_labs_itsm { syslog(ip("192.168.50.20") transport("udp")); }; destination d_local { file("/var/log/messages_${HOST}"); }; destination d_ss { snmp( host("192.168.50.10") version("v2c") community("public") trap-obj('.1.3.6.1.6.3.1.1.4.1.0', 'Objectid', '.1.3.6.1.4.1.18372.3.1.1.1.2.1') snmp-obj('.1.3.6.1.4.1.18372.3.1.1.1.1.1.0', 'Octetstring', '${MESSAGE}') snmp-obj('.1.3.6.1.4.1.18372.3.1.1.1.1.2.0', 'Octetstring', 'admin') snmp-obj('.1.3.6.1.4.1.18372.3.1.1.1.1.3.0', 'Ipaddress', '${HOST}') ); }; log { source(s_labs_itsm); destination(d_local); destination(d_ss); }; Can anyone help me out understanding what I'm doing wrong? Thanks! -- Armando Mártires

2 1

Syslog-ng header & rfc5424
by Florian Goulais 19 Jul '19

19 Jul '19

Hi. I'am trying to understand why syslog-ng and an header (timestamp host) to log as the rfc specify another header. Someone can explain me the reason plz ? Regards Florian

3 6

Get local log files written immediately even if remote log server is unresponsive
by Jim Segrave 16 Jul '19

16 Jul '19

We are running syslog-ng 3.16.1 on Centos 7.4.1708 on a central logging host. We have a large (nearly 1,000) servers also running the same version of syslog-ng on the same Centos release. The servers are configured to log locally and also forward logs to the central logging host. This morning we encountered a problem - syslog-ng was running on the logging host, but was not processing incoming logs or locally generated ones. The servers forwarding to the central host did not write anything to their local log files, a small but significant portion of them had syslog crash, after which is was restarted by systemd, but still no logs were written until syslog-ng was forcibly stopped on the central server and then restarted. Connections to the central server weren't failing in the sense of TCP close or reset, but logs were accumulating on all the servers, including the central one, in the cache file for buffering logs. For our purposes, we need to have up-to-the moment logs available on the individual servers, so an admin going in to troubleshoot on a server who only has console access still has recent logs to consult if needed. Is there a way to tell syslog-ng to write local logs immediately even if it's currently buffering logs for sending to a non-responsive remote server?

3 3

Missing messages
by Klimek, Bryan J. 16 Jul '19

16 Jul '19

First time poster, so be gentle. We run syslog-ng in our environment as a centralized syslog manager for the archiving of syslog data. We have over 2000+ Unix/Linux systems sending their syslog data with a daily ingest rate of about 10GB per day. It was recently pointed out to me that one particular message from one specific host is not getting persisted to the files on our syslog-ng server all the time. That is to say, it is intermittent. If one person can find one message that is not making into our syslog-ng archive, I can only assume that we are dropping other messages as well. How can I debug if and when messages are being lost and not making into the files on my centralized syslog server? Bryan Klimek Mayo Clinic

4 10

Cannot send Syslog-ng to Elasticsearch
by Allen Olivas 12 Jul '19

12 Jul '19

Hello, Recently I've tried following along with the Syslog-NG to Elasticsearch and Kibana blog posts and Admin Documentation for integrating Syslog-NG into Elasticsearch but I'm unable to integrate the two. I see in the .conf files the destination calls for creating and Index Pattern for Syslog-NG but when I curl the existing indices I do not see syslog-ng. Also, I'm now receiving two errors. The first I'm fairly certain we need to resolve but I've not been able to find adequate documentation on how to identify the issue let along resolve it, and the second I'm not sure if we actually need to fix. The two issues: Issue#1: I/O error occurred syslog-ng[26432]: Syslog connection established; fd='12', server='AF_INET(127.0.0.1:9200)', local='AF_INET(0.0.0.0:0)' syslog-ng[26432]: I/O error occurred while writing; fd='12', error='Broken pipe (32)' syslog-ng[26432]: Syslog connection broken; fd='12', server='AF_INET(127.0.0.1:9200)', time_reopen='60' Issue#2: Error opening plugin module; module='mod-java', error='libjvm.so: cannot open shared object file: No such file or directory' For issue 1 I'm not sure what to do or how to resolve it. For issue 2, I know for certain libjvm does exist, and I've mapped the LD_LIBRARY_PATH to the directory libjvm.so resides in. Ultimately, are these two issues preventing Syslog-NG from sending to Elasticsearch or are they just separate issues to tackle after I get things cleared up, and most importantly if they're not related, how do I integrate Syslog-NG with Elasticsearch and Kibana. Documentation is not helpful and not concise. Thanks!

4 18

Periodic "Destination timed out, reaping" and disconnection on a write fifo (pipe)
by Charles Eidsness 27 Jun '19

27 Jun '19

I kind of feel like this is going to be a dumb question but I just can't figure this out. I'm trying to get syslog-ng working with sagan, but every 60 seconds syslog-ng closes the fifo it's writing to, which closes the sagan connection to the fifo. When running in verbose mode syslog-ng is throwing up a "Destination timed out, reaping..." message. I simplified my syslog-ng configuration to: source s_sys { system(); }; destination d_fifo { pipe( "/tmp/test.fifo" ); }; log { source(s_sys); destination(d_fifo); }; And then run use cat to read the fifo: cat /tmp/test.fifo It sends all of the log traffic every but every 60 seconds cat closes. Is this expected behavior? Am I missing something? I'm using syslog-ng version 3.19.1. Thanks! Charles == Charles Eidsness Chief Engineer *CCX Technologies* (613)791-9715

2 7

Sending Messages to Slack via a HTTP Proxy
by Frank Crawford 26 Jun '19

26 Jun '19

We have recently upgraded to syslog-ng v3.19 so we can use the slack hook, but have an issue that to access the URL we will need to go through a squid web proxy. How do we configure the details of the proxy in either syslog-ng or in the slack destination? Regards Frank -- [http://www.ac3.com.au/emailsigs/v2/ac3_logo.png]<http://www.ac3.com.au> the human touch in a technical world Frank Crawford Technology and Infrastructure Manager t 02 9199 0995 | m 0401 714 870 e frank.crawford(a)ac3.com.au<mailto:frank.crawford@ac3.com.au> Level 8 East, 8 Central Ave, Eveleigh NSW 2015 [http://www.ac3.com.au/emailsigs/v2/linkedin.png] <https://www.linkedin.com/company/ac3> [http://www.ac3.com.au/emailsigs/v2/twitter.png]<https://twitter.com/ac3ptyltd> [http://www.ac3.com.au/emailsigs/v2/iso.png]<http://www.ac3.com.au/about-us/news/double-iso-certification-ac3> This email is for intended recipients only. If you are not an intended recipient, you must not disclose, copy or distribute this email to others or act upon the information contained in this email in any way. This email may contain privileged or confidential information. If you have received this email in error, please notify the sender and then delete it from your system.

2 4

error on syslog-ng elasticsearch2 destination
by joao reis 26 Jun '19

26 Jun '19

Hi, I'm getting the following error when trying to configure the elasticseach2 backend: Error parsing destination statement, destination plugin java not found in block destination elasticsearch2() at /usr/share/syslog-ng/include/scl/elasticsearch/plugin.conf:25:4:3-4:7: 1 2 #Start Block block destination elasticsearch2() at /usr/share/syslog-ng/include/scl/elasticsearch/plugin.conf:25 3 4-----> java( 4-----> ^^^^ 5 class_path("//usr/lib64/syslog-ng/java-modules/*.jar://usr/lib64/syslog-ng/java-modules/elastic-jest-client/*.jar:/*.jar") 6 class_name("org.syslog_ng.elasticsearch_v2.ElasticSearchDestination") 7 option("index", "logserver1") 8 option("type", "syslog") 9 option("server", "localhost") This is the block that causes the error: destination d_es { elasticsearch2 ( cluster("logserver1") client-mode("http") index("logserver1") type("syslog") template("$(format-json --scope rfc5424 --scope nv-pairs --exclude DATE --key ISODATE)") ); }; root@elk-server1 conf.d]# rpm -qa|grep syslog-ng syslog-ng-3.21.1-3.el7.x86_64 syslog-ng-java-3.21.1-3.el7.x86_64 syslog-ng-http-3.21.1-3.el7.x86_64 Any ideia? Thanks in advance.

1 1

syslog-ng-3.22: unofficial Debian/Ubuntu packages
by Laszlo Budai 25 Jun '19

25 Jun '19

Hi, syslog-ng 3.22.1[1] packages are available in OBS repo[2]. List of supported OSs: * Debian 8.0 * Debian 9.0 [including armv7l] * Raspbian 9.0 * Ubuntu 14.04 * Ubuntu 16.04 * Ubuntu 18.04 * Ubuntu 18.10 Install ------- example: Debian 9.0 1. get release key wget -qO - http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… | sudo apt-key add - 2. add repo to APT sources eg.: /etc/apt/sources.list.d/syslog-ng-obs.list deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… ./ Then `apt-get update` and `apt-get install syslog-ng-core` Links -------- [1] https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.22.1 [2] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… regards, Laszlo Budai

1 0

syslog-ng-3.22.1
by Laszlo Budai 25 Jun '19

25 Jun '19

3.22.1 Highlights * Sending SNMP traps: Using the new snmp() destination, incoming log messages can be converted to SNMP traps, as the fields of the SNMP messages can be customized with macros. (#2693<https://github.com/balabit/syslog-ng/pull/2693>) * $(template) dynamic binding: Extends the $(template) template function to allow dynamic binding. For example, the name of the template to be invoked can come from the message (name-value pairs). (#2716<https://github.com/balabit/syslog-ng/pull/2716>) * syslog(), network(): Add dynamic-window-size() option to enable dynamic flow control that distributes the specified amount of window between active connections at runtime. This can be used in low-memory environments, where only a small subset of the active clients sends messages at high rate. (#2772<https://github.com/balabit/syslog-ng/pull/2772>) Features * match(): Add support for the template() option (#2715<https://github.com/balabit/syslog-ng/pull/2715>) * add-contextual-data(): Allow using templates in name-value pairs (#2711<https://github.com/balabit/syslog-ng/pull/2711>) * Add support for floating point operations in template functions (#2742<https://github.com/balabit/syslog-ng/pull/2742>) * Add support for usec precision when parsing time (#2709<https://github.com/balabit/syslog-ng/pull/2709>) Bugfixes * Fix null pointer access when destinations are suspended (#2778<https://github.com/balabit/syslog-ng/pull/2778>) * Fix grouping-by() deadlock (#2758<https://github.com/balabit/syslog-ng/pull/2758>) * Fix a general source-related crash and enhance wildcard-file()'s bookmark handling (#2589<https://github.com/balabit/syslog-ng/pull/2589>) * Fix infinite loop (reload/reopen) (#2739<https://github.com/balabit/syslog-ng/pull/2739>) * Fix python() package/module name collision (#2438<https://github.com/balabit/syslog-ng/pull/2438>) * Fix escaped quote in block argument (#2781<https://github.com/balabit/syslog-ng/pull/2781>) * Reintroduce test on SYSLOG_NG_HAVE_TIMEZONE (#2774<https://github.com/balabit/syslog-ng/pull/2774>) * snmp(): Fix template leak (#2746<https://github.com/balabit/syslog-ng/pull/2746>) Other changes * Never drop flow-controlled messages: The meaning of log-fifo-size() has changed to avoid dropping flow-controlled messages when log-fifo-size() is misconfigured. From now on, log-fifo-size() only affects messages that are not flow-controlled. (#2753<https://github.com/balabit/syslog-ng/pull/2753>) * The -d/--debug syslog-ng command line flag no longer implies -e/--stderr. If you want to redirect internal() source to stderr, use the -e/--stderr option explicitly. (#2731<https://github.com/balabit/syslog-ng/pull/2731>) * dbld, RPM and DEB packaging improvements (#2724<https://github.com/balabit/syslog-ng/pull/2724>) * Checkpoint parser improvements (#2740<https://github.com/balabit/syslog-ng/pull/2740>) * Reset the timezone on config reload event (#2691<https://github.com/balabit/syslog-ng/pull/2691>) * geoip2(): Include IP into the error message (#2743<https://github.com/balabit/syslog-ng/pull/2743>) * Improve regexp error messages (#2796<https://github.com/balabit/syslog-ng/pull/2796>) * http(): Warn if less workers used than urls (#2757<https://github.com/balabit/syslog-ng/pull/2757>) * http(): Allow URLs to be specified by a space/comma separated string (#2699<https://github.com/balabit/syslog-ng/pull/2699>) * loggen: Change message rate at runtime using signals (#2756<https://github.com/balabit/syslog-ng/pull/2756>) * debun: add acquire_running_syslog_config function (#2752<https://github.com/balabit/syslog-ng/pull/2752>) * FreeBSD fixes for the test suite (#2783<https://github.com/balabit/syslog-ng/pull/2783>) Notes to the developers * ivykis: update to 0.42.4 (#2736<https://github.com/balabit/syslog-ng/pull/2736>) * Support generator plugins in global options (#2747<https://github.com/balabit/syslog-ng/pull/2747>) * logthrfetcher: new constants (#2766<https://github.com/balabit/syslog-ng/pull/2766>) * logthrsourcedrv: support position tracking (#2750<https://github.com/balabit/syslog-ng/pull/2750>) * Light: Support pre-commit and tox (#2725<https://github.com/balabit/syslog-ng/pull/2725>) * Enable Bison error flags: conflicts-sr/rr (#2762<https://github.com/balabit/syslog-ng/pull/2762>) * Dynamic stats constant registration (#2761<https://github.com/balabit/syslog-ng/pull/2761>) Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Andras Mitzki, Antal Nemes, Attila Szakacs, Balazs Scheidler, Christian Michallek, Fabien Wernli, Gabor Nagy, Kyeong Yoo, Laszlo Budai, Laszlo Szemere, László Várady, Mehul Prajapati, Norbert Takacs, Oleksii Hamov, Péter Kókai, Romain Tartière, Zoltan Pallagi. https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.22.1 [https://avatars1.githubusercontent.com/u/986981?s=400&v=4]<https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.22.1> balabit/syslog-ng<https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.22.1> syslog-ng is an enhanced log daemon, supporting a wide range of input and output methods: syslog, unstructured text, queueing, SQL & NoSQL. - balabit/syslog-ng github.com

1 0