- syslog-ng - lists.balabit.hu

MS-SQL Question
by Garridan 07 Feb '19

07 Feb '19

Hello! I'm a new syslog-ng user, so please be gentle with me. :) I'm attempting to log to an MS-SQL database and would like to send to different tables in the same DB based on the source IP - for example device A to its own table, device B to its own table, and so on. I thought I would simply need to create the same destination but define different table names in each, it works, however under load the syslog-ng service starts restarting over and over and eventually MS-SQL errors and alerts that the login packet is structurally invalid. Is it possible to send to different table names in this manner or is there another way to do it? Thanks!

6 8

syslog-ng 3.19 not processing /dev/log on RH#L6
by Evan Rempel 29 Jan '19

29 Jan '19

We just had a network even in our environment that resulted in many hosts being isolated on the network in some very odd ways. The hosts had some connections inbound, but no outbound and the network seemed to come and go very quickly. syslog-ng detected that it lost connection to its down-stream syslog collectors a did the normal connection reaping. We have this set for 5 seconds. At some point the syslog-ng process terminated and the supervisor process launched a new child. This worked well, but in this odd senario this happened quickly and at on point the supervisor process failed to reap one of its defunct children but still launched a new child process. The defunct process seemed to hold onto the /dev/log handle so the new child could not get access to it and the result is that we lost hours of logs that flow through the /dev/log socket. When we discovered the issue, there where two children of the supervisor process. One sas in a "defunct" state while the other was running, but only logging kernel messages (from /dev/kmsg) and from the internal syslog-ng source. I suspect that the child reaping process on SIGCHLD does not use a while loop and misses some children. Evan.

2 3

syslog-ng survey
by Scheidler, Balázs 29 Jan '19

29 Jan '19

Hi, If you have a bit of a time, please help us by filling this survey about syslog-ng: https://www.surveymonkey.com/r/ZHJ9MHP It is really appreciated. Cheers, Bazsi

1 0

Running multiple syslog-ng instances on the same host (RHEL6 CO6 SL6)
by Evan Rempel 20 Jan '19

20 Jan '19

In my environment we want to have isolation between the standard OS based syslog-ng service and the central syslog-ng collector service, which of course is running on a host that already has an OS based syslog-ng process running on it. Running muliple syslog-ng daemons under systemd is not so difficult. Copy the system service file and systemd automatically registers a .d folder for custom configurations, however, there is nothing automatic about the System V init mechanism used in the RHEL6 based systems. Don't worry because they syslog-ng init script from the copr yum repository (https://copr.fedorainfracloud.org/coprs/czanik) starting at 3.19 has already thought of this and made multiple instances as automatic as they can be. Quick steps to create a second syslog-ng instance: 1. Decide on the new name extension (shown here is yourName) 2. Copy the included init script cp /etc/init.d/syslog-ng /etc/init.d/syslog-ng-yourName 3. Change the SYSLOGNG_SUFFIX value in your new init script to the value "-yourName". You need the leading hyphen because your added -yourName to the init file name. 4. Copy and customize the included customization file cp /etc/sysconfig/syslog-ng /etc/sysconfig/syslog-ng-yourName By default all of the configuration and runtime files will default to file names including the -yourName suffix, so very little if any customization should be needed. 5. Create a new syslog-ng configuration file for your new instance vi /etc/syslog-ng/syslog-ng-yourName 6. Enable the auto-start of your new instance chkconfig --add syslog-ng-yourName 7. Start your instance /sbin/service syslog-ng-yourName start Thanks again to all of the people adding value to an already great product!

1 0

Issues with sql driver
by N. Max Pierson 18 Jan '19

18 Jan '19

Hi List, I have 2 questions about the sql driver. First, I am trying to get messages into sql using the sql driver but I get an error when I try and restart syslog-ng when I enable the log statement with the sql destination. The syslog-ng --syntax-only command runs without any issues but systemd throws and error that it cannot restart the service but doesn't give a clear reason. My config is below, doesn't anyone know where in a log I can see why it won't restart?? source s_network { udp(ip(0.0.0.0) port(514)); }; destination d_mysql { sql(type(mysql) host("127.0.0.1") username("syslog-ng") password("password") database("syslog") table("messages_${HOST}") columns("date", "host", "message") values("${R_DATE}", "${HOST}", "${MESSAGE}") indexes("date", "host") ); }; log { source(s_network); destination(d_mysql); }; My second question is can you use a template with the sql destination driver? I need to reformat some Cisco Nexus logs because of how it formats the date (looks to be non RFC compliant) and if so, can someone give me a sample of config with the template in the sql destination driver? I cannot seem to find in the docs if this is even possible much less and example of how to do it. TIA, Max

3 10

Insider 2019-01: central server; survey; SNMP; Slack; Conferences & Webinars
by Czanik, Péter 17 Jan '19

17 Jan '19

Dear syslog-ng users, This is the 72nd issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Creating a central syslog server -------------------------------- Your home network might already contain some devices or systems like a home server, a WiFi router, a media player, or home automation system. Creating a central syslog server to store system logs or syslogs of these systems is considered a best practice. Learn, how to get started: https://balagetech.hu/en/2019/01/14/creating-a-central-syslog-server/ Looking for your feedback about syslog-ng ----------------------------------------- Help us set the direction for syslog-ng by providing your valuable feedback through this short, 9-question survey. Your input will be of great value in determining which features and capabilities to focus our efforts on, going forward. While this survey can be completed anonymously, we would appreciate if you would share your email address, in case we would need to clarify your response and input. https://www.surveymonkey.com/r/ZHJ9MHP SNMP traps to Splunk -------------------- As we learned at the Splunk .conf18 in October, forwarding SNMP traps to Splunk can be a challenging task. Luckily, using syslog-ng can simplify it for us. All we need to make sure about is that snmptrapd logs traps to a file. The syslog-ng application can read and parse that file and forward the traps to Splunk HEC (HTTP Event Collector). https://www.syslog-ng.com/community/b/blog/posts/using-syslog-ng-to-forward… Send your log messages to Slack ------------------------------- One of the many ways members of a team collaborate is to use Slack. >From syslog-ng version 3.19, you can send log messages to Slack. You can receive critical log messages in real-time in your Slack client on your mobile or desktop. Learn how to set up alerting to Slack in just a few minutes. https://www.syslog-ng.com/community/b/blog/posts/send-your-log-messages-to-… CONFERENCES * FOSDEM talk: Extending syslog-ng in Python ( https://fosdem.org/2019/schedule/track/python/ ) You can also discuss syslog-ng with me at the following events: * https://devconf.info/cz * https://wiki.centos.org/Events/Dojo/Brussels2019 * https://cfgmgmtcamp.eu/ WEBINARS Coming up: * Send logs using the Advanced Log Transport Protocol (ALTP): https://www.syslog-ng.com/event/send-logs-using-the-advanced-log-transport-… You can also watch our past webinars: * Log ingestion to Splunk HEC: https://www.brighttalk.com/webcast/16207/338190 * High performance log streaming to HDFS with syslog-ng: https://www.brighttalk.com/webcast/16207/335943 Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Rewrite regex not working entirely
by N. Max Pierson 17 Jan '19

17 Jan '19

Hi List, I am using version 3.5 and it seems as though regex (posix or pcre) doesn't work completely. Take the example string below (which is the message part of the syslog). Jan 15 15:50:57 CST: %DAEMON-3-SYSTEM_MSG: NTP Receive dropping message: Received NTP control mode packet. Drop count:147972 - ntpd[15029] I am trying to match the date at the beginning of the message and remove it. When I use \w, \s, \d, etc, they do not match anything. If I match on a character classes it works fine (ex [a-z]+ or [0-9]+). Here is my statement for the rewrite rule. rewrite r_nexus{ subst("^[a-z]+ [0-9]+ [0-9]+:[0-9]+:[0-9]+ [a-z]+: ", "", value("MESSAGE"), type("posix"), flags("ignore-case"), condition(filter(f_nexus))); }; The above seems to get me what I want but are the character matches not supposed to work in syslog-ng version 3.5?? Regards, Max

5 6

visualizing firewall logs
by Czanik, Péter 15 Jan '19

15 Jan '19

Hi, It was almost two years ago, when I first published how to visualize firewall logs on a map and create a movie out of the results. By that time talking to many syslog-ng users the consensus was, that it has no practical value. It is an eye candy. It can only be used to attract C-level guys attentions towards security. Tow years later I see people asking me about log visualization a lot more often. Did anything change? Is it already more than an eye candy? I posted an Twitter poll about the topic: https://twitter.com/PCzanik/status/1084804811713929216 Let me know your opinion with a vote, and if you have the time also with an explanation here or on Twitter. Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Syslog-ng newbie.
by Jason Long 30 Dec '18

30 Dec '18

Hello.I want to send Nginx log via syslog-ng to a server that Elasticsearch and Kibana installed on it. I'm a beginner and searched in Google but can't find a good tutorial. I want to know on Nginx server with syslog-ng installed is below config enough? options { chain_hostnames(off); flush_lines(0); use_dns(no); use_fqdn(no); owner("root"); group("adm"); perm(0640); stats_freq(0); bad_hostname("^gconfd$");};source s_myfile { file("/var/log/nginx/access.log" follow_freq(1) flags(no-parse));};log { source(s_myfile); }; I'm thankful if anyone share advice. Thank you.

2 3

Syslog-ng and NetSkope JSON logs
by Garcia, Julio (InfoSec) 21 Dec '18

21 Dec '18

Hi, I’m trying to get a valid json formatted log file. I’ve tried several options but none of them seem to work or parse out the data correctly. Here’s what the syslog-ng config looks like. @version:3.14 @include "scl.conf" # syslog-ng configuration file. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # # Note: it also sources additional configuration files (*.conf) # located in /etc/syslog-ng/conf.d/ ###################################### #Sources ###################################### source s_netskope { tcp(ip(0.0.0.0) port(51410)); }; ###################################### #Destinations ###################################### destination d_netskope { file("/data/log/syslog/netskope/$HOST/$YEAR-$MONTH-$DAY-netskope.log" create_dirs(yes)); }; ###################################### # Filters ###################################### ###################################### # Log ###################################### log { source( s_netskope); destination(d_netskope); }; options { flush_lines (0); time_reopen (10); log_fifo_size (1000); chain_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); owner("user01"); group("user01"); dir-owner("user01"); dir-group("user01"); dir-perm(0755); perm(0755); }; Any help is greatly appreciated. Thank you, Julio Garcia Pro, Information Security Engineer CoreLogic Direct (949) 214-1284 Mobile (714) 474-5254 julgarcia(a)corelogic.com<mailto:julgarcia@corelogic.com> corelogic.com<http://www.corelogic.com/> | Blog<http://www.corelogic.com/blog/default.aspx> LinkedIn<http://www.linkedin.com/company/corelogic> | Twitter<http://twitter.com/corelogicinc> | Facebook<http://www.facebook.com/CoreLogic> | Google+<https://plus.google.com/114618839782139347829> Our Vision: Deliver unique property-level insights that power the global real estate economy ****************************************************************************************** This message may contain confidential or proprietary information intended only for the use of the addressee(s) named above or may contain information that is legally privileged. If you are not the intended addressee, or the person responsible for delivering it to the intended addressee, you are hereby notified that reading, disseminating, distributing or copying this message is strictly prohibited. If you have received this message by mistake, please immediately notify us by replying to the message and delete the original message and any copies immediately thereafter. Thank you. ****************************************************************************************** CLLD

3 4

Adding a background thread
by Naveen Revanna 21 Dec '18

21 Dec '18

Hello syslog-ng developers, I am planning to add a new thread of execution to syslog-ng. I am seeking inputs/expert advice for the same. Motivation: I had previously discussed a problem here where syslog-ng would hang if dns server is unreachable. The reason this was happening was that getaddrinfo() would take some time to timeout and all syslog-ng tasks would pile up resulting in delayed execution which would appear that syslog-ng has hung. I took the approach of using getaddrinfo_a() which would do the async resolution of the address. I have recently come to understand that getaddrinfo_a() finally calls getaddrinfo() in a separate pthread that it spawns. This is causing some issues resulting in a crash. The stacktrace is pointing to problem in glib allocators. Here is what I am thinking. I will call iv_thread() to create a new thread as part of the main process. All this thread does is execute a fixed task (which in my case is to just do address resolution). I am looking for inputs from you developers for pointers as to what would be a good place to spawn this thread. This thread would do very minimal stuff and hope not to have much complexity. Are there things that I should look out for if we have another thread of execution. Do any of you think it is bad idea and time consuming process to introduce a new thread to existing syslog-ng. I have some understanding of the code and have read a bit about ivykis and some of it's functions. All I'll be bothering you will be for your expertise and would be doing the development on my side. Sorry if I am bothering you in your time with family. Happy Holidays :) Thanks, Naveen.

1 0

Cyxtera json file and syslog-ng
by Garcia, Julio (InfoSec) 20 Dec '18

20 Dec '18

Hi, I’m currently sending Cyxtera json logs to our syslog-ng server via syslog-ng but for some reason I’m not able to get all of the data to parse out correctly. It’s being parsed out with _json. I was at some point able to get the message in json form but all the data was within a MESSAGE bracket. This is where I’m at now. Looks like it parses correctly but it’s all under _json. Is there a way to remove that and keep everything else? This is what my syslog-ng looks like. @version:3.14 @include "scl.conf" # syslog-ng configuration file. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # # Note: it also sources additional configuration files (*.conf) # located in /etc/syslog-ng/conf.d/ ###################################### #Sources ###################################### source s_cyxtera { tcp(port(51404) flags(no-parse)); }; ###################################### #Destinations ###################################### destination d_cyxtera { file("/data/log/syslog/cyxtera/$HOST/$YEAR-$MONTH-$DAY-cyxtera.log" template("$(format-json --scope dot-nv-pairs)\n")); }; ###################################### # Parser ###################################### parser p_json { json-parser (prefix(".json.")); }; ###################################### # Log ###################################### log { source( s_cyxtera); parser(p_json); destination(d_cyxtera); }; options { flush_lines (0); time_reopen (10); log_fifo_size (1000); chain_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); owner("usr01"); group("usr01"); dir-owner("user01"); dir-group("user01"); dir-perm(0755); perm(0755); }; This is how the data is showing up in splunk. I had to clear some info but these are the main keys showing up. I have to expand _json: to get everything inside the key. I want it where I don’t have the _json and everything else shows as their individual key. Hope this makes sense. I know I’m using some of the wrong terminology. [cid:image001.png@01D49851.505CD0E0] Thank you, Julio Garcia Pro, Information Security Engineer CoreLogic Direct (949) 214-1284 Mobile (714) 474-5254 julgarcia(a)corelogic.com<mailto:julgarcia@corelogic.com> corelogic.com<http://www.corelogic.com/> | Blog<http://www.corelogic.com/blog/default.aspx> LinkedIn<http://www.linkedin.com/company/corelogic> | Twitter<http://twitter.com/corelogicinc> | Facebook<http://www.facebook.com/CoreLogic> | Google+<https://plus.google.com/114618839782139347829> Our Vision: Deliver unique property-level insights that power the global real estate economy ****************************************************************************************** This message may contain confidential or proprietary information intended only for the use of the addressee(s) named above or may contain information that is legally privileged. If you are not the intended addressee, or the person responsible for delivering it to the intended addressee, you are hereby notified that reading, disseminating, distributing or copying this message is strictly prohibited. If you have received this message by mistake, please immediately notify us by replying to the message and delete the original message and any copies immediately thereafter. Thank you. ****************************************************************************************** CLLD

2 2

ISODATE Macro not working when message contains non-ascii chars
by Jose Angel Santiago 20 Dec '18

20 Dec '18

Hi, I'm using syslog-ng 3.18.1, and I'm having some issues due to my special use case. Unfortunately my syslog-ng agent reads some binary files which are piped through the log file I'm processing and sending it to elasticsearch. When this happens, the elastic bulk post fail (So I'am affected by this issue https://github.com/balabit/syslog-ng/issues/1993) I've debugged it and the problem was that message generated when applying the template is empty. This is my network destination: destination d_syslog_tcp { network( "syslog-relay.node" port(7601) template("${ISODATE} ${HOST} ${CONTAINER-IP} ${MESSAGE}\n") template-escape(no) disk-buffer( mem-buf-size(200M) disk-buf-size(500M) reliable(yes) dir("/var/syslog-ng") ) ); }; I've opened a netcat server as syslog-relay.node and piped a small binary file through the stdout file which I'm using as source, and this what the relay receives: 2018-12-17T15:35:57+00:00 agent-4 192.168.54.17 � 2018-12-17T15:35:57+00:00 agent-4 192.168.54.17 3F 2018-12-17T15:35:57+00:00 agent-4 192.168.54.17 $ 2018-12-17T15:35:57+00:00 agent-4 192.168.54.17 4) As you can see, fourth message has no timestamp, hostname or ip address, it only contains "$"char, so when this message is sent to elastic, I've got a mapperException since it tries to validate the received timestamp. I know I can customize elastic so it doesn't fail in case a wrong timestamp is received, but the point is that syslog-ng seems to fail when templating the message, doesn't it? Also it would be great if syslog-ng had any mechanism to filter all those binary messages, I've already tried with filter expression, excluding all non-printable chars, but the issue still happens. Regards. -- | Jose Angel Santiago [image: Logo_signature2.png] <http://www.stratio.com/> Vía de las dos Castillas, 33, Ática 4, 3ª Planta 28224 Pozuelo de Alarcón, Madrid, Spain +34 918 286 473 | www.stratio.com <https://twitter.com/stratiobd> <https://www.linkedin.com/company/stratiobd> <https://www.youtube.com/c/StratioBD>

2 4

3.19.1: help to spread the word
by Czanik, Péter 14 Dec '18

14 Dec '18

Hi, As you have seen here on the mailing list, syslog-ng 3.19.1 was released yesterday. Help us the spread the word about this new release where you can. Here are some pointers for you: - list of changes and source tgz: https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.19.1 - unofficial packages for: * Debian / Ubuntu: https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… * Fedora / RHEL / CentOS: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng319/ * openSUSE / SLES: https://build.opensuse.org/project/show/home:czanik:syslog-ng319 Tweets, LinkedIn posts, etc. based on your own experiences are of course the best, but here is also a list of posts you can easily share with a click of a button: - announcement: * https://twitter.com/PCzanik/status/1073233850581614592 * https://www.linkedin.com/feed/update/urn:li:activity:6479000458223587328 - packages: * deb: https://twitter.com/Budai_L/status/1073237929445781504 * suse: https://twitter.com/PCzanik/status/1073483538341740544 * fedora/red hat: https://twitter.com/PCzanik/status/1073506635195154432 Thanks, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

2 1

unofficial 3.19.1 rpm packages
by Czanik, Péter 14 Dec '18

14 Dec '18

Hi, Unofficial syslog-ng 3.19.1 rpm packages are now available: - openSUSE / SLES: https://build.opensuse.org/project/show/home:czanik:syslog-ng319 - Fedora / RHEL / CentOS: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng319/ Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Insider 2018-12: 3.19 release; optimizing Splunk; Python source; HTTP batch;
by Czanik, Péter 13 Dec '18

13 Dec '18

Dear syslog-ng users, This is the 71st issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Version 3.19 of syslog-ng released ---------------------------------- Version 3.19 of syslog-ng has been released with plenty of new features and bugfixes. Performance of the HTTP destination improved thanks to load-balancing to multiple servers. You can use this to send the messages to a set of ingestion nodes or indexers of your SIEM solution if a single node cannot handle the load. The new Slack destination allows you to send alerts to a Slack channel. Read the complete list of changes at https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.19.1. Optimize your Splunk infrastructure using new syslog-ng features ------------------------------------------------------- Learn how to use less resources for better performance in Splunk! Many people have been using syslog-ng for decades without knowing that it receives new features as well as bugfixes. While many Linux utilities are practically in maintenance mode, syslog-ng keeps evolving constantly. A strong focus in recent years has been on message parsing and destination drivers. After my talk at Suricon, Splunk users explained how they will change their syslog-ng configurations to optimize their Splunk infrastructure. https://www.syslog-ng.com/community/b/blog/posts/optimize-your-splunk-infra… Python source in syslog-ng -------------------------- Using syslog-ng 3.18 and newer releases, you can write new source drivers for syslog-ng in Python. While performance is not as good as C, you gain flexibility and ease of implementation. There are quite a few log sources without a ready to use C API, but with a Python API. Using the Python source of syslog-ng you can leverage these. https://www.syslog-ng.com/community/b/blog/posts/python-source-in-syslog-ng Bulk mode message sending to Elasticsearch with syslog-ng http() destination --------------------------------------------------------- Learn how to send log messages in bulk mode to your Elasticsearch server with syslog-ng. Bulk mode offers better performance, because it sends multiple log messages in a single POST request. https://www.syslog-ng.com/community/b/blog/posts/bulk-mode-message-sending-… WEBINARS You can watch our past webinars: * Log ingestion to Splunk HEC: https://www.brighttalk.com/webcast/16207/338190 * High performance log streaming to HDFS with syslog-ng: https://www.brighttalk.com/webcast/16207/335943 Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

3 2

Unofficial syslog-ng-3.19 Debian/Ubuntu packages
by Laszlo Budai 13 Dec '18

13 Dec '18

Hi, syslog-ng 3.19.1[1] packages are available in OBS repo[2]. List of supported OSs: * Debian 8.0 * Debian 9.0 [armv7l is still building] * Ubuntu 14.04 * Ubuntu 16.04 * Ubuntu 16.10 * Ubuntu 17.04 * Ubuntu 17.10 * Ubuntu 18.04 * Ubuntu 18.10 Install ------- example: Debian 9.0 1. get release key wget -qO - http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… | sudo apt-key add - 2. add repo to APT sources eg.: /etc/apt/sources.list.d/syslog-ng-obs.list deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… ./ Then `apt-get update` and `apt-get install syslog-ng-core` Links -------- [1] https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.19.1 [2] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… regards, Laszlo Budai

1 0

OSE 3.19.1
by Nemes, Antal 13 Dec '18

13 Dec '18

3.19.1 Features - HTTP load balancer (#2347 <https://github.com/balabit/syslog-ng/pull/2347>) - Slack destination (#2451 <https://github.com/balabit/syslog-ng/pull/2451>) - Add Cisco Catalyst formatted triplets support to cisco-parser() (#2394 <https://github.com/balabit/syslog-ng/pull/2394>) - Add RFC5424 syslog support to the system() source (FreeBSD 12.0 support) (#2430 <https://github.com/balabit/syslog-ng/pull/2430>) - Add network interface() option to network sources (#2389 <https://github.com/balabit/syslog-ng/pull/2389>) - Add so-reuseport() to network drivers (#2379 <https://github.com/balabit/syslog-ng/pull/2379>) - Enable supporting HTTP redirects (#2136 <https://github.com/balabit/syslog-ng/pull/2136>) Bugfixes - Fix frequent disconnects of syslog() with TLS (#2432 <https://github.com/balabit/syslog-ng/pull/2432>) - Fix possible refcount leak during reload/shutdown (#2434 <https://github.com/balabit/syslog-ng/pull/2434>) - Fix message storm on trace level (#2425 <https://github.com/balabit/syslog-ng/pull/2425>) - Fix use after free in file destinations (time-reap) (#2418 <https://github.com/balabit/syslog-ng/pull/2418>) - Fixing a few memleaks in the Java destination (#2417 <https://github.com/balabit/syslog-ng/pull/2417>) - Fix telegram dst default ca dir (#2416 <https://github.com/balabit/syslog-ng/pull/2416>) - Fix prefix handling in $(list-concat) and $(strip) (#2405 <https://github.com/balabit/syslog-ng/pull/2405>) - Fixing an eventfd leak with ivykis<=0.38 (threaded destinations) (#2404 <https://github.com/balabit/syslog-ng/pull/2404>) - Process flush result after worker thread exits (threaded destinations) (#2402 <https://github.com/balabit/syslog-ng/pull/2402>) - hdfs: do not try to write unopened file (#2391 <https://github.com/balabit/syslog-ng/pull/2391>) - Fix leaks in redis() destination (#2383 <https://github.com/balabit/syslog-ng/pull/2383>) - Block location tracking fixes (#2378 <https://github.com/balabit/syslog-ng/pull/2378>) - Fix $(basename) and $(dirname) in the presence of a prefix (#2371 <https://github.com/balabit/syslog-ng/pull/2371>) - Fixing a false positive corruption detection in non-reliable diskq ( #2356 <https://github.com/balabit/syslog-ng/pull/2356>) - Check if /proc/kmsg can be opened in system-source (#2408 <https://github.com/balabit/syslog-ng/pull/2408>) - Fix include guard in systemd-journal (#2445 <https://github.com/balabit/syslog-ng/pull/2445>) - Remove hexadecimal and octal number parsing from templates (#2401 <https://github.com/balabit/syslog-ng/pull/2401>) Other changes - Do not load certs from default CURL ca-dir by default (http() destination) (#2410 <https://github.com/balabit/syslog-ng/pull/2410>) - Disable SSL compression by default (#2372 <https://github.com/balabit/syslog-ng/pull/2372>) - Flush lines cleanup (#2386 <https://github.com/balabit/syslog-ng/pull/2386>, #2392 <https://github.com/balabit/syslog-ng/pull/2392>) - Refine json-parser() log messages to be less alarming (#2437 <https://github.com/balabit/syslog-ng/pull/2437>) - Move some messages to trace (#2358 <https://github.com/balabit/syslog-ng/pull/2358>) - Make include-path more discoverable (#2426 <https://github.com/balabit/syslog-ng/pull/2426>) - Adding build flag -Wmissing-format-attribute and -Wsuggest-attribute=noreturn (#2423 <https://github.com/balabit/syslog-ng/pull/2423>) - Rewrite filter unit tests based on criterion (#2422 <https://github.com/balabit/syslog-ng/pull/2422>) - PytestFramework in Travis (#2415 <https://github.com/balabit/syslog-ng/pull/2415>) - syslog-ng-mod-java debian pkg should depend on headless jre (#2388 <https://github.com/balabit/syslog-ng/pull/2388>) - Add contextual data error reporting improvements & csv-scanner refactor (#2373 <https://github.com/balabit/syslog-ng/pull/2373>) - Afsocket remove unused functions/bitfields (#2363 <https://github.com/balabit/syslog-ng/pull/2363>) - Afsocket minor cleanup/refactor (#2355 <https://github.com/balabit/syslog-ng/pull/2355>) - Riemann worker (#2313 <https://github.com/balabit/syslog-ng/pull/2313>) - Afsql threaded dest driver (#2097 <https://github.com/balabit/syslog-ng/pull/2097>) - dbld: do not mount .gitconfig if missing (#2419 <https://github.com/balabit/syslog-ng/pull/2419>) - dbld: Add missing docbook-xsl packages (#2398 <https://github.com/balabit/syslog-ng/pull/2398>) - dbld: update criterion to 2.3.3 (#2396 <https://github.com/balabit/syslog-ng/pull/2396>) - dbld: Remove "proposed" Ubuntu repository from enable_dbgsyms() (#2382 <https://github.com/balabit/syslog-ng/pull/2382>) - dbld: Add new target "list-builder-images" (#2381 <https://github.com/balabit/syslog-ng/pull/2381>) - dbld: Support Ubuntu Bionic and update existing images (#2318 <https://github.com/balabit/syslog-ng/pull/2318>) - dbld: release target should use the default image (#2464 <https://github.com/balabit/syslog-ng/pull/2464>) Notes to the developers - PytestFramework: Add initial test framework (#1940 <https://github.com/balabit/syslog-ng/pull/1940>) Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Abder Benbachir, Andras Mitzki, Antal Nemes, Attila Szakacs, Balazs Scheidler, Gabor Nagy, Gergely Tonté, JP Vossen, Juhasz Viktor, Laszlo Budai, Laszlo Szemere, László Várady, Norbert Takacs, Peter Kokai, Zoltan Pallagi. https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.19.1 <https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.17.1> balabit/syslog-ng <https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.19.1> syslog-ng is an enhanced log daemon, supporting a wide range of input and output methods: syslog, unstructured text, queueing, SQL & NoSQL. github.com Regards, Antal Nemes

1 0

Re: [syslog-ng] syslog-ng not working with selinux in enforcing mode
by Mohan Lal 11 Dec '18

11 Dec '18

Hi, I am trying to configure syslog-ng to run with selinux in enforcing mode. I have tried to configure ports for syslog-ng in linux using semanage command using following link:- https://www.syslog-ng.com/community/b/blog/posts/using-syslog-ng-with-selin… under section -> Using a different port a=After that I rebooted the server, I can see syslog-ng is listening in all ports configured in /etc/syslog-ng/conf.d/*.conf ( using netstat -nap|grep ports). I can see all ports in output of 'semanage port --list|grep syslog' also. but it's not processing any data. It start processing data the moment I disable selinux in /etc/selinux/config file. what might be the problem please, how do I go about troubleshooting this. syslog-ng --version syslog-ng 3.5.6 Installer-Version: 3.5.6 Revision: Compile-Date: Dec 30 2015 19:57:24 Available-Modules: affile,afprog,afsocket-notls,afsocket-tls,afsocket,afstomp,afuser,basicfuncs,confgen,cryptofuncs,csvparser,dbparser,linux-kmsg-format,syslogformat,system-source Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-Linux-Caps: on Enable-Pcre: on cat /etc/redhat-release Red Hat Enterprise Linux Server release 7.4 (Maipo) Regards Mohan This e-mail message (including any attachment) is intended only for the personal use of the recipient(s) named above. This message is confidential and may be legally privileged. If you are not an intended recipient, you may not review, copy or distribute this message. If you have received this communication in error, please notify us immediately by e-mail and delete the original message. Any views or opinions expressed in this message are those of the author only. Furthermore, this message (including any attachment) does not create any legally binding rights or obligations whatsoever, which may only be created by the exchange of hard copy documents signed by a duly authorised representative of Hutchison 3G UK Limited. Hutchison 3G UK Limited is a company registered in England and Wales with company number 3885486. Registered Office Star House, 20 Grenfell Road, Maidenhead, Berkshire SL6 1EH

1 0

balabit/syslog-ng:latest container and kafka output
by Mathias Dufresne 30 Nov '18

30 Nov '18

Hi all, I'm trying to push logs from files to Kafka using syslog-ng container provided there: https://hub.docker.com/r/balabit/syslog-ng/ I used also : https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edi… The ERROR: When running the container including Kafka configuration I get: [2018-11-30T12:20:51.324773] WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; [2018-11-30T12:20:51.478631] Can't find class; class_name='org.syslog_ng.kafka.KafkaDestination' [2018-11-30T12:20:51.478667] If you have reloaded syslog-ng, the JVM is not used anymore, but it is still running. If you want to stop JVM, stop syslog-ng and then start syslog-ng again; [2018-11-30T12:20:51.478680] Error initializing message pipeline; plugin_name='java', location='#buffer:4:3' Here are my configuration files: /etc/syslog-ng/syslog-ng.conf : @version: 3.18 @include "scl.conf" # Syslog-ng configuration file, compatible with default Debian syslogd # installation. # First, set some global options. options { chain_hostnames(off); flush_lines(0); #use_dns(yes); #use_fqdn(yes); use_dns(no); use_fqdn(no); #owner("root"); #group("adm"); perm(0640); # The default action of syslog-ng is to log a STATS line # to the file every 10 minutes. That's pretty ugly after a while. # Change it to every 12 hours so you get a nice daily update of # how many messages syslog-ng missed (0). stats_freq(0); stats_level(1); # The default action of syslog-ng is to log a MARK line # to the file every 20 minutes. That's seems high for most # people so turn it down to once an hour. Set it to zero # if you don't want the functionality at all. mark_freq(0); bad_hostname("^gconfd$"); threaded(yes); log_msg_size (16384); log_fifo_size (8000000); # # Time management: # # The goal is to have in logs the original date and time set by # the syslog which effectively receives the message # and that this timestamp is not changed during it's journey # to reach the database. # # To achieve that all clients are configured with keep_timestamp(no) # for syslog-ng marks when it receives the message. # This is done globally because system() and internal() sources are not # both supporting this keep_timestamp option when the syslog() source # used for networked syslog support that option. This options is forced to "yes" # in remote source(). keep_timestamp(no); # # Time precision is increased to microseconds frac_digits(6); # And format is set to "iso": YYYY-MM-DDThh:mm:ss.0123456+02.00 # where "T" is the separator between date and time # and +02.00 GMT decay (as an example) ts_format("iso"); }; ######################## # Sources ######################## # This is the default behavior of sysklogd package # Logs may come from unix stream, but not from another machine. # source s_src { system(); internal(); }; # Log from any hosts around # source s_net { network( transport("tcp") port(601) flags("syslog-protocol", "threaded") keep_timestamp(yes) max-connections(250) log_iw_size(25000) log_fetch_limit(100) flags("threaded") ); }; source s_app_logs { file(/opt/test.log,flags(no-parse),follow_freq(1),keep-timestamp(yes)); }; ################################################################################################# # Destinations ################################################################################################# destination d_messages { file("/var/log/messages"); }; destination d_clients { file("/var/log/syslog_clients/$HOST"); }; destination d_clients_all { file("/var/log/syslog_clients/all_logs"); }; @include "kafka_destination.conf" ################################################################################################# # Destinations ################################################################################################# log { source(s_net); # destination(d_clients); }; log { source(s_src); # destination(d_messages); }; ################################################################################################# # Filters ################################################################################################# filter f_no_debug { not level(debug); }; --------------------------------------------------------------------------------------------- kafka_destination.conf: @module mod-java @define java-module-dir "/usr/lib/syslog-ng/3.18/java-modules:/usr/lib/syslog-ng/3.18/java-modules/:/usr/lib/syslog-ng/3.18/java-modules/*.jar" destination d_kafka { kafka( client-lib-dir("/usr/lib/syslog-ng/3.18/java-modules:/usr/lib/syslog-ng/3.18/java-modules/:/usr/lib/syslog-ng/3.18/java-modules/*.jar:/etc/syslog-ng/kafka_libs/libs/:/etc/syslog-ng/kafka_libs/libs/*.jar:/etc/syslog-ng/kafka_libs/libs") #client-lib-dir("/usr/lib/syslog-ng/3.18/java-modules/:/usr/lib/x86_64-linux-gnu/") kafka-bootstrap-servers("10.11.12.129:9093,10.11.12.130:9093, 10.11.12.131:9093,10.11.12.132:9093,10.11.12.133:9093,10.11.12.134:9093") topic("roma") properties-file("/etc/syslog-ng/kafka_producer.properties") ); }; log { source(s_app_logs); destination(d_kafka); }; --------------------------------------------------------------------------------------------- Directory /etc/syslog-ng/kafka_libs/libs is in fact "libs" directory from kafka_2.11-2.1.0.tgz which comes from http://kafka.apache.org/downloads. Here is the content of this libs directory : ls kafka_libs/libs/ -1 activation-1.1.1.jar aopalliance-repackaged-2.5.0-b42.jar argparse4j-0.7.0.jar audience-annotations-0.5.0.jar commons-lang3-3.5.jar compileScala.mapping compileScala.mapping.asc connect-api-2.1.0.jar connect-basic-auth-extension-2.1.0.jar connect-file-2.1.0.jar connect-json-2.1.0.jar connect-runtime-2.1.0.jar connect-transforms-2.1.0.jar guava-20.0.jar hk2-api-2.5.0-b42.jar hk2-locator-2.5.0-b42.jar hk2-utils-2.5.0-b42.jar jackson-annotations-2.9.7.jar jackson-core-2.9.7.jar jackson-databind-2.9.7.jar jackson-jaxrs-base-2.9.7.jar jackson-jaxrs-json-provider-2.9.7.jar jackson-module-jaxb-annotations-2.9.7.jar javassist-3.22.0-CR2.jar javax.annotation-api-1.2.jar javax.inject-1.jar javax.inject-2.5.0-b42.jar javax.servlet-api-3.1.0.jar javax.ws.rs-api-2.1.1.jar javax.ws.rs-api-2.1.jar jaxb-api-2.3.0.jar jersey-client-2.27.jar jersey-common-2.27.jar jersey-container-servlet-2.27.jar jersey-container-servlet-core-2.27.jar jersey-hk2-2.27.jar jersey-media-jaxb-2.27.jar jersey-server-2.27.jar jetty-client-9.4.12.v20180830.jar jetty-continuation-9.4.12.v20180830.jar jetty-http-9.4.12.v20180830.jar jetty-io-9.4.12.v20180830.jar jetty-security-9.4.12.v20180830.jar jetty-server-9.4.12.v20180830.jar jetty-servlet-9.4.12.v20180830.jar jetty-servlets-9.4.12.v20180830.jar jetty-util-9.4.12.v20180830.jar jopt-simple-5.0.4.jar kafka_2.11-2.1.0.jar kafka_2.11-2.1.0.jar.asc kafka_2.11-2.1.0-javadoc.jar kafka_2.11-2.1.0-javadoc.jar.asc kafka_2.11-2.1.0-scaladoc.jar kafka_2.11-2.1.0-scaladoc.jar.asc kafka_2.11-2.1.0-sources.jar kafka_2.11-2.1.0-sources.jar.asc kafka_2.11-2.1.0-test.jar kafka_2.11-2.1.0-test.jar.asc kafka_2.11-2.1.0-test-sources.jar kafka_2.11-2.1.0-test-sources.jar.asc kafka-clients-2.1.0.jar kafka-log4j-appender-2.1.0.jar kafka-streams-2.1.0.jar kafka-streams-examples-2.1.0.jar kafka-streams-scala_2.11-2.1.0.jar kafka-streams-test-utils-2.1.0.jar kafka-tools-2.1.0.jar log4j-1.2.17.jar lz4-java-1.5.0.jar maven-artifact-3.5.4.jar metrics-core-2.2.0.jar osgi-resource-locator-1.0.1.jar plexus-utils-3.1.0.jar reflections-0.9.11.jar rocksdbjni-5.14.2.jar scala-library-2.11.12.jar scala-logging_2.11-3.9.0.jar scala-reflect-2.11.12.jar slf4j-api-1.7.25.jar slf4j-log4j12-1.7.25.jar snappy-java-1.1.7.2.jar validation-api-1.1.0.Final.jar zkclient-0.10.jar zookeeper-3.4.13.jar zstd-jni-1.3.5-4.jar ---------------------------------------------------------------------------------------------- Here is "syslog-ng --version" inside the conainer: root@e1ca5fdc292a:/# syslog-ng --version syslog-ng 3 (3.18.1) Config version: 3.18 Installer-Version: 3.18.1 Revision: 3.18.1-1 Compile-Date: Oct 24 2018 10:04:41 Module-Directory: /usr/lib/syslog-ng/3.18 Module-Path: /usr/lib/syslog-ng/3.18 Available-Modules: add-contextual-data,afamqp,affile,afmongodb,afprog,afsmtp,afsocket,afsql,afstomp,afuser,appmodel,basicfuncs,cef,confgen,cryptofuncs,csvparser,date,dbparser,disk-buffer,examples,geoip-plugin,geoip2-plugin,graphite,hook-commands,http,json-plugin,kvformat,linux-kmsg-format,map-value-pairs,mod-java,mod-python,pacctformat,pseudofile,redis,riemann,sdjournal,snmptrapd-parser,stardate,syslogformat,system-source,tags-parser,tfgetent,xml Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-Linux-Caps: on Enable-Systemd: on root@e1ca5fdc292a:/# mod-java is there but that's not enough and I can"t understand what I am missing. ------------------------------------------- Packages: root@e1ca5fdc292a:/# dpkg -l | grep kafka ii librdkafka-dev:amd64 0.8.5-2 amd64 library implementing the Apache Kafka protocol (development headers) ii librdkafka1:amd64 0.8.5-2 amd64 library implementing the Apache Kafka protocol ii syslog-ng-mod-kafka 3.18.1-1 amd64 Enhanced system logging daemon (Kafka destination) root@e1ca5fdc292a:/# If information are missing, please tell me. If you have some clue, that would be really appreciated. Best regards, mathias

1 0

Verifying local logs and remote logs are being sent to destination graylog
by Rodney Bizzell 23 Nov '18

23 Nov '18

I want to make sure that my local logs on my syslog box are being sent to my graylog host what config changes do I need to make to ensure that is configured correctly. Can I have an example for illustration. Thanks. @version:3.5 @include "scl.conf" # syslog-ng configuration file. # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # # Note: it also sources additional configuration files (*.conf) # located in /etc/syslog-ng/conf.d/ options { flush_lines (0); time_reopen (10); log_fifo_size (250000); chain_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { system(); internal(); # udp(ip(0.0.0.0) port(514)); }; source s_net { udp(ip(0.0.0.0) port(514)); #tcp(ip(0.0.0.0) port(514) max-connections(256)); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; destination d_graylog { tcp("graylog.host" port (12201) ); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; log { source(s_net); destination(d_graylog); }; log { source(s_sys); filter(f_default); destination(d_graylog);}; # Source additional configuration files (.conf extension only) @include "/etc/syslog-ng/conf.d/*.conf"

4 20

IPV6 and IPV4 syslog-ng issue
by venkateswarlu vinjamuri 22 Nov '18

22 Nov '18

Hi, *Issue:* *when configured both IPV4 and IPV6 syslog server IP's, I could see syslog-ng messages going to only one syslog server, either IPV4 or IPV6 syslog-ng server and that too to the latest server IP configured only.* Is there any known issue ? Please help me on this. Thanks & Regards, Venkat.

2 3

Syslog-ng open source usage.
by vijay amruth 14 Nov '18

14 Nov '18

Hello team, does open source versions use is any different after syslog-ng acquisition? Vijay Amrut. -- Thanks, Vijay Amrut.

3 2

Global flush-lines option VS elasticsearch2 destination flush-limit param?
by Jose Angel Santiago 11 Nov '18

11 Nov '18

Hi, It seems both parameters are made for the same purpose, but I guess they don't. ¿Could you explain to me what is the difference between these two parameters? Thanks in advance. -- | Jose Angel Santiago [image: Logo_signature2.png] <http://www.stratio.com/> Vía de las dos Castillas, 33, Ática 4, 3ª Planta 28224 Pozuelo de Alarcón, Madrid, Spain +34 918 286 473 | www.stratio.com <https://twitter.com/stratiobd> <https://www.linkedin.com/company/stratiobd> <https://www.youtube.com/c/StratioBD>

2 2

sending log messages to graylog
by Rodney Bizzell 06 Nov '18

06 Nov '18

I got syslog to send and echo test message from syslog server to my graylog box. How do I ensure that my syslog box will send other servers logs to my graylog box through my syslog server. I am going to setup ipvsadm as load-balancer to point my legacy application to my syslog server and then they should get shipped through to graylog. Any information is greatly appreciated

4 16

log drop-and-delete directory
by Nik Ambrosch 06 Nov '18

06 Nov '18

Looking to create a drop-off directory that syslog-ng handles instead of needing to execute in a separate script.. flow would go something like this: 1) mv file.log /syslog-tmp/ 2) syslog-ng reads /syslog-tmp/file.log 3) syslog-ng deletes /syslog-tmp/file.log when done consuming Sounds simple but I can't seem to figure out a good way to do this. The other option is to read file with a script, send out with logger (or whatever), and hope that syslog-ng is running & healthy. Thanks.

6 10

Incremental update of syslog-ng config
by Monitto Pitchaimani sebastin (mopitcha) 05 Nov '18

05 Nov '18

Hi All, Is there a way to update the syslog-ng config without restarting the syslog-ng? Regards, Monitto

4 5

Unable to make syslog-ng listen to port 514
by Fosiul Alam 05 Nov '18

05 Nov '18

Hello i have installed syslog-ng in our ubuntu server with version : 3.13 its running fine, I am using the default syslog-ng file and i have change this line as per documentation source s_net { syslog(ip(192.168.1.19) transport("tcp")); }; but its not listing on port 514 What Do i have to do make it listen ? Thanks for the help ● syslog-ng.service - System Logger Daemon Loaded: loaded (/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2018-11-02 18:22:46 UTC; 4min 5s ago Docs: man:syslog-ng(8) Main PID: 16470 (syslog-ng) Tasks: 1 (limit: 2361) CGroup: /system.slice/syslog-ng.service └─16470 /usr/sbin/syslog-ng -F Nov 02 18:22:46 syslog systemd[1]: Starting System Logger Daemon... Nov 02 18:22:46 syslog systemd[1]: Started System Logger Daemon. -- Regards Fosiul Alam

7 12

unofficial syslog-ng 3.18 rpm packages
by Czanik, Péter 25 Oct '18

25 Oct '18

Hi, All my unofficial syslog-ng 3.18 rpm packages are now on-line. You can learn about what is new or what changed in this release in the release announcement: https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.18.1 Note, that not all features are available in all packages. Changes in packaging: - openSUSE & SLES: no changes Available at: https://build.opensuse.org/project/show/home:czanik:syslog-ng318 - Fedora & RHEL/CentOS 7: Python support changed from 2.7 to 3.X, where X is the Python 3 version available in the base distribution. For EPEL 7 packages it means 3.4, even if EPEL has 3.6. Available at: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng318/ - RHEL/CentOS 6: no spoof source support (does not compile), no MonogDB support (source no more bundled with syslog-ng and no ready to use external package is available). Let me know if you miss either of these features. Available at: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng318epel6/ Installation instructions are available at https://www.syslog-ng.com/community/b/blog/posts/installing-latest-syslog-n… but note, that you need to change the version number from 3.14 to 3.18 in the examples. Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Unofficial syslog-ng-3.18 packages for Debian/Ubuntu
by Laszlo Budai 24 Oct '18

24 Oct '18

Hi, syslog-ng 3.18.1[1] packages are available in OBS repo[2]. List of supported OSs: * Debian 8.0 * Debian 9.0 [armv7l is still building] * Ubuntu 14.04 * Ubuntu 16.04 * Ubuntu 16.10 * Ubuntu 17.04 * Ubuntu 17.10 * Ubuntu 18.04 * and Ubuntu 18.10 Install ------- example: Debian 9.0 1. get release key wget -qO - http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… | sudo apt-key add - 2. add repo to APT sources eg.: /etc/apt/sources.list.d/syslog-ng-obs.list deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… ./ Then `apt-get update` and `apt-get install syslog-ng-core` Links -------- [1] https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.18.1 [2] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… regards, Laszlo Budai

1 0

parsing JSON logs just won't work
by Michael Niemand 18 Oct '18

18 Oct '18

Hi, I can’t get JSON parsing to work. I’ve consulted the documentation and Google but with no luck. I have an app, that puts out simple json log messages like: {"level":"error","message":"connection ended without disconnect receipt","timestamp":"2018-10-12T17:49:08.650Z"} All I want to do, is parse these 3 values and send them to a hosted Graylog cluster. Sending works, but the message gets inserted as application name: {"level" message: "error","message":"connection ended without disconnect receipt","timestamp":"2018-10-12T17:49:08.650Z"} it's almost like syslog-ng doesn't even interpret the file as json. I tried different variants but I am at my wits end now... This is my config (on the application host; it should send the logs directly to the logging cluster) @version: 3.5 @include "scl.conf" @include "`scl-root`/system/tty10.conf" options { chain_hostnames(off); flush_lines(0); use_dns(no); use_fqdn(no); owner("root"); group("adm"); perm(0640); stats_freq(0); bad_hostname("^gconfd$"); }; source s_src { file( “/var/log/worker/error.log" flags(no-parse) ); }; template unitManagerTemplate { template("$(format-json --scope dot-nv-pairs) [sdid@123456 X-OVH-TOKEN=\"XXXXXXXXXXXXXXXXXXXXXXXXXX\"\n"); }; destination ovhPaaSLogs { tcp("gra2.logs.ovh.com" port(6514), template(unitManagerTemplate), ts_format("iso"), tls(peer-verify("require-trusted") ca_dir("/etc/ssl/certs/")), keep-alive(yes), so_keepalive(yes), ); }; parser p_json { json-parser(prefix(".json.")); }; log { source(s_src); parser(p_json); destination(ovhPaaSLogs); }; @include "/etc/syslog-ng/conf.d/" I also tried a different a template variant like this: template("${.json.level} ${.json.message} ${.json.timestamp} [sdid@123456 X-OVH-TOKEN=\"XXXXXXXXXXXXXXXXXXXXXXXXXX\"\n”); I also tried parsing the messages as text: template("{\"level\":\"${PRIORITY}\",\"message\":\"${MSG}\",\"timestamp\":\"${ISODATE}\"} - [sdid@32473 X-OVH-TOKEN=\"XXXXXXXXXXXXXXXXXXXXXXXXXX\" pid=\"${PID}\" facility=\"${FACILITY}\" priority=\"${PRIORITY}\"] ${MSG}\n"); What shows up in Graylog is absolutely identical (like described in the beginning). In fact, every variant that I tried changed absolutely nothing. The conf.d folder is empty though. I’d appreciate any help! Best regards, Michael

3 3

Insider 2018-10: 3.18 release; Splunk .conf18; Python destination; Python parser;
by Czanik, Péter 18 Oct '18

18 Oct '18

Dear syslog-ng users, This is the 70th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Version 3.18 of syslog-ng released ------------------------------------- Version 3.18 of syslog-ng has been released with plenty of new features and bugfixes. Performance of the HTTP destination improved thanks to batching and multi-threading. The new Python source allows you to create source drivers for syslog-ng in Python. Learn about the small, but important changes at https://www.syslog-ng.com/community/b/blog/posts/small-is-beautiful-what-s-… or read the complete list of changes at https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.18.1. syslog-ng at Splunk .conf18 --------------------------- Balázs Scheidler, original author of syslog-ng, gave a talk at the annual Splunk conference. His talk about simplifying complex log data ingestion with syslog-ng is now available on-line. Learn about the new features coming up in syslog-ng to make it an even better choice to feed Splunk with log data: https://conf.splunk.com/files/2018/recordings/simplifying-complex-log-data-… Python destination ------------------ You can send your log messages to many different destinations using syslog-ng, but of course not everywhere. This is where the Python destination of syslog-ng can come handy. You can extend syslog-ng easily with your own code written in Python and store your log messages to the destination of your choice. Learn how to develop your own Python code for syslog-ng from our blog series, starting with the very basics and showing a complete working MQTT destination implemented in Python at the end: https://www.syslog-ng.com/community/b/blog/posts/python-destination-getting… https://www.syslog-ng.com/community/b/blog/posts/python-destination-getting… https://www.syslog-ng.com/community/b/blog/posts/writing-python-destination… Python parser ------------- The Python parser of syslog-ng not only enables you to parse any type of log message, but you can also use it to enrich messages. From this blog you will learn how to extract information from a specially formatted log message, and how to create new name-value pairs by consulting external databases about data contained in your log messages. I will show these using two simple parsers: one resolves host names from IP addresses, the other one uses regular expressions to parse synthetic log messages from the loggen utility. https://www.syslog-ng.com/community/b/blog/posts/parsing-log-messages-with-… WEBINARS You can watch our past webinars: * Supercharge your syslog-ng parsing: https://www.brighttalk.com/webcast/16207/330198 * High performance log streaming to HDFS with syslog-ng: https://www.brighttalk.com/webcast/16207/335943 Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

OSE 3.18.1
by Nemes, Antal 17 Oct '18

17 Oct '18

3.18.1 Features - HTTP batching (#2135 <https://github.com/balabit/syslog-ng/pull/2135>) - HTTP multi threading (#2291 <https://github.com/balabit/syslog-ng/pull/2291>) - HTTP framing support for batches (#2190 <https://github.com/balabit/syslog-ng/pull/2190>) - Python source (#2308 <https://github.com/balabit/syslog-ng/pull/2308>) - Template support for Python destination (#2196 <https://github.com/balabit/syslog-ng/pull/2196>) - HDFS time-reap (#2257 <https://github.com/balabit/syslog-ng/pull/2257>) - Riemann batching (#2098 <https://github.com/balabit/syslog-ng/pull/2098>) - flush-timeout() for HTTP (#2251 <https://github.com/balabit/syslog-ng/pull/2251>) - Base64 encoding template function (#2223 <https://github.com/balabit/syslog-ng/pull/2223>) - url-decode() template function (#2278 <https://github.com/balabit/syslog-ng/pull/2278>) - Allow IPv4 mapped addresses in IPv6 mode (#2312 <https://github.com/balabit/syslog-ng/pull/2312>) - app-parser: set ${.app.name} to the application identified (#2290 <https://github.com/balabit/syslog-ng/pull/2290>) - Value pairs improvements (#2287 <https://github.com/balabit/syslog-ng/pull/2287>) - syslog-ng-ctl config to print pre-processed configuration (#2280 <https://github.com/balabit/syslog-ng/pull/2280>) - Add support for whitespace in template functions (#2271 <https://github.com/balabit/syslog-ng/pull/2271>) - Verify the configuration is the same as on the disk (#2345 <https://github.com/balabit/syslog-ng/pull/2345>) Bugfixes - Fix queue counters race condition (#2316 <https://github.com/balabit/syslog-ng/pull/2316>) - AMQP configurable max connection and frame size (#2343 <https://github.com/balabit/syslog-ng/pull/2343>) - Fix wakeup in threaded sources (#2339 <https://github.com/balabit/syslog-ng/pull/2339>) - Fix libnet memory leak in network destinations (#2331 <https://github.com/balabit/syslog-ng/pull/2331>) - Fix unexpected flag check-hostname in syslog-parser (#2314 <https://github.com/balabit/syslog-ng/pull/2314>) - Fix memory leak in dbparser (#2311 <https://github.com/balabit/syslog-ng/pull/2311>) - Inline Python code comment generates syntax error (#2319 <https://github.com/balabit/syslog-ng/pull/2319>) - Password protected SSL keys portability (MADV_DONTDUMP) (#2341 <https://github.com/balabit/syslog-ng/pull/2341>) - Fixing compiler warnings from armv7l (#2301 <https://github.com/balabit/syslog-ng/pull/2301>) - Riemann crashes in flush (#2296 <https://github.com/balabit/syslog-ng/pull/2296>) - Revert sticky hook option (#2295 <https://github.com/balabit/syslog-ng/pull/2295>) - Small stats fixes (#2294 <https://github.com/balabit/syslog-ng/pull/2294>) - Detect filter loop (#2283 <https://github.com/balabit/syslog-ng/pull/2283>, #2288 <https://github.com/balabit/syslog-ng/pull/2288>) - Fix infinite loop in threaded destinations with ivykis prior 0.39 ( #2275 <https://github.com/balabit/syslog-ng/pull/2275>) - Fix log expr node use after free (#2269 <https://github.com/balabit/syslog-ng/pull/2269>) - Remove gradle from the list of "BuildRequires" (RPM packaging) (#2266 <https://github.com/balabit/syslog-ng/pull/2266>) - Fix wildcard-source memleak when directory removed (#2261 <https://github.com/balabit/syslog-ng/pull/2261>, #2267 <https://github.com/balabit/syslog-ng/pull/2267>) - Missing macros: C_AMPM, C_USEC, C_MSEC, C_HOUR12 (#2259 <https://github.com/balabit/syslog-ng/pull/2259>) - Fix cisco timestamp parsing (#2272 <https://github.com/balabit/syslog-ng/pull/2272>) - Undefined filter reference (#2273 <https://github.com/balabit/syslog-ng/pull/2273>) - Fix the literal() type hint (#2286 <https://github.com/balabit/syslog-ng/pull/2286>) - logwriter, affile, afsocket: fixing "internal overflow". (#2250 <https://github.com/balabit/syslog-ng/pull/2250>) - lib/gsockaddr.c: modify the unix salen calculation (#2285 <https://github.com/balabit/syslog-ng/pull/2285>) - dbld: fix dbld/rules deb failure (#2282 <https://github.com/balabit/syslog-ng/pull/2282>) - Update data type to avoid conversion. (#2281 <https://github.com/balabit/syslog-ng/pull/2281>) - compat/getent: add support for platforms that lack the r versions ( #2244 <https://github.com/balabit/syslog-ng/pull/2244>) - Fix memory leak caused by saving stats counter to persist config (#2279 <https://github.com/balabit/syslog-ng/pull/2279>) - Hdfs: disable archive when append-enabled is configured (#2235 <https://github.com/balabit/syslog-ng/pull/2235>) - scl: add linux-audit() SCL to make files (#2230 <https://github.com/balabit/syslog-ng/pull/2230>) - DebianPackaging: Add linux-audit SCL to included dirs (#2254 <https://github.com/balabit/syslog-ng/pull/2254>) - cap_syslog capability detection (#2227 <https://github.com/balabit/syslog-ng/pull/2227>) - LogProto partial write (#2194 <https://github.com/balabit/syslog-ng/pull/2194>) Other changes - telegram, urlencode: api changes (#2299 <https://github.com/balabit/syslog-ng/pull/2299>) - python: include python2/3 in plugin description (#2337 <https://github.com/balabit/syslog-ng/pull/2337>) - Stats prepare for multiple queues per destdrv (#2302 <https://github.com/balabit/syslog-ng/pull/2302>) - Update deprecated use of tcp()/udp() to network(). This addresses #2322 <https://github.com/balabit/syslog-ng/issues/2322> (#2326 <https://github.com/balabit/syslog-ng/pull/2326>) - Message about not supported cap_syslog only at debug level (#2333 <https://github.com/balabit/syslog-ng/pull/2333>) - Few test leak fix (#2323 <https://github.com/balabit/syslog-ng/pull/2323>) - warning elimination: pointer arithmetics (#2305 <https://github.com/balabit/syslog-ng/pull/2305>) - templates: get rid off the args_lock (#2289 <https://github.com/balabit/syslog-ng/pull/2289>) - Ack tracker small refact (#2277 <https://github.com/balabit/syslog-ng/pull/2277>) - ElasticSearchDestination: Display deprecated warning message about us… (#2274 <https://github.com/balabit/syslog-ng/pull/2274>) - Improve Readme.md header structure (#2258 <https://github.com/balabit/syslog-ng/pull/2258>) - Rewrite json tests based on criterion (#2255 <https://github.com/balabit/syslog-ng/pull/2255>) - Rewrite dbparser tests based on criterion (#2252 <https://github.com/balabit/syslog-ng/pull/2252>) - Processed timestamp (#2243 <https://github.com/balabit/syslog-ng/pull/2243>) - msg parameters: remove last NULL parameter from msg macros (#2242 <https://github.com/balabit/syslog-ng/pull/2242>) - Fix threaded destination test cases (#2236 <https://github.com/balabit/syslog-ng/pull/2236>) - dbld: missing packages, deps changes (#2232 <https://github.com/balabit/syslog-ng/pull/2232>, #2332 <https://github.com/balabit/syslog-ng/pull/2332>, #2327 <https://github.com/balabit/syslog-ng/pull/2327>, #2260 <https://github.com/balabit/syslog-ng/pull/2260>, #2256 <https://github.com/balabit/syslog-ng/pull/2256>) - Use msg_trace() when emitting trace information (#2226 <https://github.com/balabit/syslog-ng/pull/2226>) Notes to the developers - Threaded source and fetcher (#2247 <https://github.com/balabit/syslog-ng/pull/2247>) - "Examples" module (#2248 <https://github.com/balabit/syslog-ng/pull/2248>) - Do not ship mongo-c-driver with syslog-ng (remove as submodule) (#2245 <https://github.com/balabit/syslog-ng/pull/2245>) Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Andras Mitzki, Antal Nemes, Balazs Scheidler, Gabor Nagy, Gergely Tonté, Laszlo Budai, Laszlo Szemere, László Várady, Maurice T. Meyer, Mahmoud Salama, Norbert Takacs, Peter Czanik, Peter Gyorko, Peter Kokai, Robert Fekete, Terez Nemes, Tibor Bodnar, Zoltan Pallagi, y-l-i. https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.18.1 <https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.17.1> balabit/syslog-ng <https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.17.1> syslog-ng is an enhanced log daemon, supporting a wide range of input and output methods: syslog, unstructured text, queueing, SQL & NoSQL. github.com Regards, Antal Nemes

1 0

syslog-ng 3.5
by Rodney Bizzell 15 Oct '18

15 Oct '18

I have install syslog-ng 3.5 and I am configuring it to send all logs to Graylog but once changes are made syslog-ng is failing. I change the address of graylog server to 0.0.0.0 @version:3.5 @include "scl.conf" # syslog-ng configuration file. # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # # Note: it also sources additional configuration files (*.conf) # located in /etc/syslog-ng/conf.d/ options { flush_lines (0); time_reopen (10); log_fifo_size (1000); chain_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (yes); }; source s_sys { system(); internal(); udp(ip(0.0.0.0) port(514)); }; source s_net { udp(ip(0.0.0.0) port(514)); tcp(ip(0.0.0.0) port(514) max-connections(256)); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; destination d_graylog { tcp("0.0.0.0" port (12201) spoof_sources(yes) ); }; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; log { source(s_net); destination(d_graylog); }; log { source(s_sys); filter(f_default); destination(d_graylog);}; # Source additional configuration files (.conf extension only) @include "/etc/syslog-ng/conf.d/*.conf" # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et:

4 14

CentOS syslog-ng using tls won't start via systemd but runs fine manually (sort of)
by craig bowser 12 Oct '18

12 Oct '18

I'm been trying to figure this out for several days now. I have Centos 7 installed on two servers and I want to use syslog-ng to send logs between them securely using TLS. On both I installed syslog-ng 3.5 using this method: https://www.syslog-ng.com/community/b/blog/posts/installing-latest-syslog-n… I have been following these methods as a starting point to set up: https://www.nineproductions.com/syslog-ng-with-tls-setup-for-centralized-lo… https://www.linux.com/blog/tls-encryption-and-mutual-authentication-using-s… https://www.flyn.org/notes/logging/index.html On the receiving server, I have syslog-ng with the following config file: ---------------------------------------------------------- @version:3.5 @include "scl.conf" # syslog-ng configuration file. # # This should behave pretty much like the original syslog on RedHat. But # it could be configured a lot smarter. # # See syslog-ng(8) and syslog-ng.conf(5) for more information. # # Note: it also sources additional configuration files (*.conf) # located in /etc/syslog-ng/conf.d/ options { flush_lines (0); time_reopen (10); # log_fifo_size (1000); # chain_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (1); keep_hostname (yes); }; source s_sys { system(); internal(); # udp(ip(0.0.0.0) port(514)); }; source s_net { tcp(ip(10.1.0.2) port(5441) tls( ca_dir("/etc/syslog-ng/ca.d") key_file("/etc/syslog-ng/newCA/serverkey.pem") cert_file("/etc/syslog-ng/newCA/servercert.pem") peer_verify(optional-untrusted)) ); }; destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern"); }; destination d_mlal { usertty("*"); }; destination d_log { file("/opt/syslog/$YEAR$MONTH$DAY-logs.log" create_dirs(yes) dir_group(logstash) dir_owner(logstash) owner(logstash) group(logstash));}; filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; log { source(s_net); destination(d_log); }; # Source additional configuration files (.conf extension only) @include "/etc/syslog-ng/conf.d/*.conf" # vim:ft=syslog-ng:ai:si:ts=4:sw=4:et: ------------------------------------------------------ Here is my java verison: # java -version openjdk version "1.8.0_181" OpenJDK Runtime Environment (build 1.8.0_181-b13) OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode) This is my OS version: # cat /etc/*release CentOS Linux release 7.2.1511 (Core) NAME="CentOS Linux" VERSION="7 (Core)" ID="centos" ID_LIKE="rhel fedora" VERSION_ID="7" PRETTY_NAME="CentOS Linux 7 (Core)" ANSI_COLOR="0;31" CPE_NAME="cpe:/o:centos:centos:7" HOME_URL="https://www.centos.org/" BUG_REPORT_URL="https://bugs.centos.org/" CENTOS_MANTISBT_PROJECT="CentOS-7" CENTOS_MANTISBT_PROJECT_VERSION="7" REDHAT_SUPPORT_PRODUCT="centos" REDHAT_SUPPORT_PRODUCT_VERSION="7" CentOS Linux release 7.2.1511 (Core) CentOS Linux release 7.2.1511 (Core) If I try to start syslog-ng via systemctl, this is what happens: # systemctl start syslog-ng Job for syslog-ng.service failed because the control process exited with error code. See "systemctl status syslog-ng.service" and "journalctl -xe" for details. # systemctl status syslog-ng.service ● syslog-ng.service - System Logger Daemon Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled) Active: failed (Result: start-limit) since Fri 2018-10-12 11:57:09 EDT; 13s ago Docs: man:syslog-ng(8) Process: 5706 ExecStart=/usr/sbin/syslog-ng -F -p /var/run/syslogd.pid (code=exited, status=2) Main PID: 5706 (code=exited, status=2) Status: "Starting up... (Fri Oct 12 11:57:09 2018" Oct 12 11:57:09 logger.com systemd[1]: Unit syslog-ng.service entered failed state. Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service failed. Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service holdoff time over, scheduling restart. Oct 12 11:57:09 logger.com systemd[1]: start request repeated too quickly for syslog-ng.service Oct 12 11:57:09 logger.com systemd[1]: Failed to start System Logger Daemon. Oct 12 11:57:09 logger.com systemd[1]: Unit syslog-ng.service entered failed state. Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service failed. Oct 12 11:57:09 logger.com systemd[1]: start request repeated too quickly for syslog-ng.service Oct 12 11:57:09 logger.com systemd[1]: Failed to start System Logger Daemon. Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service failed. #journalctl -xe Oct 12 11:57:08 logger.com polkitd[818]: Registered Authentication Agent for unix-process:5693:6083335 (system bus name :1.97 [/usr/bin/pkttyagent --notify-fd 5 - Oct 12 11:57:08 logger.com systemd[1]: Listening on Syslog Socket. -- Subject: Unit syslog.socket has finished start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog.socket has finished starting up. -- -- The start-up result is done. Oct 12 11:57:08 logger.com systemd[1]: Starting Syslog Socket. -- Subject: Unit syslog.socket has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog.socket has begun starting up. Oct 12 11:57:08 logger.com systemd[1]: Starting System Logger Daemon... -- Subject: Unit syslog-ng.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has begun starting up. Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service: main process exited, code=exited, status=2/INVALIDARGUMENT Oct 12 11:57:08 logger.com systemd[1]: Failed to start System Logger Daemon. -- Subject: Unit syslog-ng.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has failed. -- -- The result is failed. Oct 12 11:57:08 logger.com systemd[1]: Unit syslog-ng.service entered failed state. Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service failed. Oct 12 11:57:08 logger.com polkitd[818]: Unregistered Authentication Agent for unix-process:5693:6083335 (system bus name :1.97, object path /org/freedesktop/Poli Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service holdoff time over, scheduling restart. Oct 12 11:57:08 logger.com systemd[1]: Starting System Logger Daemon... -- Subject: Unit syslog-ng.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has begun starting up. Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service: main process exited, code=exited, status=2/INVALIDARGUMENT Oct 12 11:57:08 logger.com systemd[1]: Failed to start System Logger Daemon. -- Subject: Unit syslog-ng.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has failed. -- -- The result is failed. Oct 12 11:57:08 logger.com systemd[1]: Unit syslog-ng.service entered failed state. Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service failed. Oct 12 11:57:08 logger.com systemd-journal[494]: Forwarding to syslog missed 3 messages. -- Subject: One or more messages could not be forwarded to syslog -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- One or more messages could not be forwarded to the syslog service -- running side-by-side with journald. This usually indicates that the -- syslog implementation has not been able to keep up with the speed of -- messages queued. Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service holdoff time over, scheduling restart. Oct 12 11:57:08 logger.com systemd[1]: Starting System Logger Daemon... -- Subject: Unit syslog-ng.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has begun starting up. Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service: main process exited, code=exited, status=2/INVALIDARGUMENT Oct 12 11:57:08 logger.com systemd[1]: Failed to start System Logger Daemon. -- Subject: Unit syslog-ng.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has failed. -- -- The result is failed. Oct 12 11:57:08 logger.com systemd[1]: Unit syslog-ng.service entered failed state. Oct 12 11:57:08 logger.com systemd[1]: syslog-ng.service failed. Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service holdoff time over, scheduling restart. Oct 12 11:57:09 logger.com systemd[1]: Starting System Logger Daemon... -- Subject: Unit syslog-ng.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has begun starting up. Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service: main process exited, code=exited, status=2/INVALIDARGUMENT Oct 12 11:57:09 logger.com systemd[1]: Failed to start System Logger Daemon. -- Subject: Unit syslog-ng.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has failed. -- -- The result is failed. Oct 12 11:57:09 logger.com systemd[1]: Unit syslog-ng.service entered failed state. Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service failed. Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service holdoff time over, scheduling restart. Oct 12 11:57:09 logger.com systemd[1]: Starting System Logger Daemon... -- Subject: Unit syslog-ng.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has begun starting up. Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service: main process exited, code=exited, status=2/INVALIDARGUMENT Oct 12 11:57:09 logger.com systemd[1]: Failed to start System Logger Daemon. -- Subject: Unit syslog-ng.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has failed. -- -- The result is failed. Oct 12 11:57:09 logger.com systemd[1]: Unit syslog-ng.service entered failed state. Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service failed. Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service holdoff time over, scheduling restart. Oct 12 11:57:09 logger.com systemd[1]: start request repeated too quickly for syslog-ng.service Oct 12 11:57:09 logger.com systemd[1]: Failed to start System Logger Daemon. -- Subject: Unit syslog-ng.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has failed. -- -- The result is failed. Oct 12 11:57:09 logger.com systemd[1]: Unit syslog-ng.service entered failed state. Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service failed. Oct 12 11:57:09 logger.com systemd[1]: start request repeated too quickly for syslog-ng.service Oct 12 11:57:09 logger.com systemd[1]: Failed to start System Logger Daemon. -- Subject: Unit syslog-ng.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has failed. -- -- The result is failed. Oct 12 11:57:09 logger.com systemd[1]: Unit syslog.socket entered failed state. Oct 12 11:57:09 logger.com systemd[1]: syslog-ng.service failed. But if I start with the command line: # /usr/sbin/syslog-ng -Fedv -p /var/run/syslogd.pid Starting to read include file; filename='/etc/syslog-ng/scl.conf', depth='1' Global value changed; define='scl-root', value='/usr/share/syslog-ng/include/scl' Global value changed; define='include-path', value='/etc/syslog-ng:/usr/share/syslog-ng/include' Starting to read include file; filename='/usr/share/syslog-ng/include/scl/system/plugin.conf', depth='2' Module loaded and initialized successfully; module='system-source' Finishing include; filename='/usr/share/syslog-ng/include/scl/system/plugin.conf', depth='2' Starting to read include file; filename='/usr/share/syslog-ng/include/scl/pacct/plugin.conf', depth='2' Reading path for candidate modules; path='//usr/lib64/syslog-ng' Reading shared object for a candidate module; path='//usr/lib64/syslog-ng', fname='affile.so', module='affile' Registering candidate plugin; module='affile', context='source', name='file', preference='0' Registering candidate plugin; module='affile', context='source', name='pipe', preference='0' Registering candidate plugin; module='affile', context='destination', name='file', preference='0' Registering candidate plugin; module='affile', context='destination', name='pipe', preference='0' Reading shared object for a candidate module; path='//usr/lib64/syslog-ng', fname='afprog.so', module='afprog' Registering candidate plugin; module='afprog', context='source', name='program', preference='0' Registering candidate plugin; module='afprog', context='destination', name='program', preference='0' Reading shared object for a candidate module; path='//usr/lib64/syslog-ng', fname='afsocket-notls.so', module='afsocket-notls' Registering candidate plugin; module='afsocket-notls', context='source', name='unix-stream', preference='0' Registering candidate plugin; module='afsocket-notls', context='destination', name='unix-stream', preference='0' Registering candidate plugin; module='afsocket-notls', context='source', name='unix-dgram', preference='0' Registering candidate plugin; module='afsocket-notls', context='destination', name='unix-dgram', preference='0' Registering candidate plugin; module='afsocket-notls', context='source', name='tcp', preference='0' Registering candidate plugin; module='afsocket-notls', context='destination', name='tcp', preference='0' Registering candidate plugin; module='afsocket-notls', context='source', name='tcp6', preference='0' Registering candidate plugin; module='afsocket-notls', context='destination', name='tcp6', preference='0' Registering candidate plugin; module='afsocket-notls', context='source', name='udp', preference='0' Registering candidate plugin; module='afsocket-notls', context='destination', name='udp', preference='0' Registering candidate plugin; module='afsocket-notls', context='source', name='udp6', preference='0' Registering candidate plugin; module='afsocket-notls', context='destination', name='udp6', preference='0' Registering candidate plugin; module='afsocket-notls', context='source', name='syslog', preference='0' Registering candidate plugin; module='afsocket-notls', context='destination', name='syslog', preference='0' Registering candidate plugin; module='afsocket-notls', context='source', name='network', preference='0' Registering candidate plugin; module='afsocket-notls', context='destination', name='network', preference='0' Reading shared object for a candidate module; path='//usr/lib64/syslog-ng', fname='afsocket-tls.so', module='afsocket-tls' Registering candidate plugin; module='afsocket-tls', context='source', name='unix-stream', preference='100' Registering candidate plugin; module='afsocket-tls', context='destination', name='unix-stream', preference='100' Registering candidate plugin; module='afsocket-tls', context='source', name='unix-dgram', preference='100' Registering candidate plugin; module='afsocket-tls', context='destination', name='unix-dgram', preference='100' Registering candidate plugin; module='afsocket-tls', context='source', name='tcp', preference='100' Registering candidate plugin; module='afsocket-tls', context='destination', name='tcp', preference='100' Registering candidate plugin; module='afsocket-tls', context='source', name='tcp6', preference='100' Registering candidate plugin; module='afsocket-tls', context='destination', name='tcp6', preference='100' Registering candidate plugin; module='afsocket-tls', context='source', name='udp', preference='100' Registering candidate plugin; module='afsocket-tls', context='destination', name='udp', preference='100' Registering candidate plugin; module='afsocket-tls', context='source', name='udp6', preference='100' Registering candidate plugin; module='afsocket-tls', context='destination', name='udp6', preference='100' Registering candidate plugin; module='afsocket-tls', context='source', name='syslog', preference='100' Registering candidate plugin; module='afsocket-tls', context='destination', name='syslog', preference='100' Registering candidate plugin; module='afsocket-tls', context='source', name='network', preference='100' Registering candidate plugin; module='afsocket-tls', context='destination', name='network', preference='100' Reading shared object for a candidate module; path='//usr/lib64/syslog-ng', fname='afsocket.so', module='afsocket' Registering candidate plugin; module='afsocket', context='source', name='unix-stream', preference='100' Registering candidate plugin; module='afsocket', context='destination', name='unix-stream', preference='100' Registering candidate plugin; module='afsocket', context='source', name='unix-dgram', preference='100' Registering candidate plugin; module='afsocket', context='destination', name='unix-dgram', preference='100' Registering candidate plugin; module='afsocket', context='source', name='tcp', preference='100' Registering candidate plugin; module='afsocket', context='destination', name='tcp', preference='100' Registering candidate plugin; module='afsocket', context='source', name='tcp6', preference='100' Registering candidate plugin; module='afsocket', context='destination', name='tcp6', preference='100' Registering candidate plugin; module='afsocket', context='source', name='udp', preference='100' Registering candidate plugin; module='afsocket', context='destination', name='udp', preference='100' Registering candidate plugin; module='afsocket', context='source', name='udp6', preference='100' Registering candidate plugin; module='afsocket', context='destination', name='udp6', preference='100' Registering candidate plugin; module='afsocket', context='source', name='syslog', preference='100' Registering candidate plugin; module='afsocket', context='destination', name='syslog', preference='100' Registering candidate plugin; module='afsocket', context='source', name='network', preference='100' Registering candidate plugin; module='afsocket', context='destination', name='network', preference='100' Reading shared object for a candidate module; path='//usr/lib64/syslog-ng', fname='afstomp.so', module='afstomp' Registering candidate plugin; module='afstomp', context='destination', name='stomp', preference='0' Reading shared object for a candidate module; path='//usr/lib64/syslog-ng', fname='afuser.so', module='afuser' Registering candidate plugin; module='afuser', context='destination', name='usertty', preference='0' Reading shared object for a candidate module; path='//usr/lib64/syslog-ng', fname='basicfuncs.so', module='basicfuncs' Registering candidate plugin; module='basicfuncs', context='template-func', name='grep', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='if', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='echo', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='length', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='substr', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='strip', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='sanitize', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='lowercase', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='uppercase', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='replace-delimiter', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='+', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='-', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='*', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='/', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='%', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='ipv4-to-int', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='indent-multi-line', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='context-length', preference='0' Registering candidate plugin; module='basicfuncs', context='template-func', name='env', preference='0' Reading shared object for a candidate module; path='//usr/lib64/syslog-ng', fname='confgen.so', module='confgen' Reading shared object for a candidate module; path='//usr/lib64/syslog-ng', fname='cryptofuncs.so', module='cryptofuncs' Registering candidate plugin; module='cryptofuncs', context='template-func', name='uuid', preference='0' Registering candidate plugin; module='cryptofuncs', context='template-func', name='hash', preference='0' Registering candidate plugin; module='cryptofuncs', context='template-func', name='sha1', preference='0' Registering candidate plugin; module='cryptofuncs', context='template-func', name='sha256', preference='0' Registering candidate plugin; module='cryptofuncs', context='template-func', name='sha512', preference='0' Registering candidate plugin; module='cryptofuncs', context='template-func', name='md4', preference='0' Registering candidate plugin; module='cryptofuncs', context='template-func', name='md5', preference='0' Reading shared object for a candidate module; path='//usr/lib64/syslog-ng', fname='csvparser.so', module='csvparser' Registering candidate plugin; module='csvparser', context='parser', name='csv-parser', preference='0' Reading shared object for a candidate module; path='//usr/lib64/syslog-ng', fname='dbparser.so', module='dbparser' Registering candidate plugin; module='dbparser', context='parser', name='db-parser', preference='0' Reading shared object for a candidate module; path='//usr/lib64/syslog-ng', fname='linux-kmsg-format.so', module='linux-kmsg-format' Registering candidate plugin; module='linux-kmsg-format', context='format', name='linux-kmsg', preference='0' Reading shared object for a candidate module; path='//usr/lib64/syslog-ng', fname='syslogformat.so', module='syslogformat' Registering candidate plugin; module='syslogformat', context='format', name='syslog', preference='0' Registering candidate plugin; module='syslogformat', context='parser', name='syslog-parser', preference='0' Reading shared object for a candidate module; path='//usr/lib64/syslog-ng', fname='system-source.so', module='system-source' Finishing include; filename='/usr/share/syslog-ng/include/scl/pacct/plugin.conf', depth='2' Starting to read include file; filename='/usr/share/syslog-ng/include/scl/syslogconf/plugin.conf', depth='2' Module loaded and initialized successfully; module='confgen' Finishing include; filename='/usr/share/syslog-ng/include/scl/syslogconf/plugin.conf', depth='2' Finishing include; filename='/etc/syslog-ng/scl.conf', depth='1' Module loaded and initialized successfully; module='afsocket-tls' Module loaded and initialized successfully; module='affile' Finishing include; content='source confgen system', depth='1' Module loaded and initialized successfully; module='afuser' Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:5] Compiling s_sys reference [source] at [/etc/syslog-ng/syslog-ng.conf:5] Compiling s_sys sequence [source] at [/etc/syslog-ng/syslog-ng.conf:2] Compiling #unnamed junction [log] at [/etc/syslog-ng/syslog-ng.conf:2] Compiling #unnamed single [log] at [#buffer:1:1] Compiling #unnamed single [log] at [#buffer:2:1] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:2] Compiling f_kernel reference [filter] at [/etc/syslog-ng/syslog-ng.conf:5] Compiling f_kernel sequence [filter] at [/etc/syslog-ng/syslog-ng.conf:4] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:4] Compiling d_kern reference [destination] at [/etc/syslog-ng/syslog-ng.conf:5] Compiling d_kern sequence [destination] at [/etc/syslog-ng/syslog-ng.conf:4] Compiling #unnamed junction [log] at [/etc/syslog-ng/syslog-ng.conf:4] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:4] Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling s_sys reference [source] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling f_default reference [filter] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling f_default sequence [filter] at [/etc/syslog-ng/syslog-ng.conf:4] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:4] Compiling d_mesg reference [destination] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling d_mesg sequence [destination] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed junction [log] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling s_sys reference [source] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling f_auth reference [filter] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling f_auth sequence [filter] at [/etc/syslog-ng/syslog-ng.conf:4] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:4] Compiling d_auth reference [destination] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling d_auth sequence [destination] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed junction [log] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling s_sys reference [source] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling f_mail reference [filter] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling f_mail sequence [filter] at [/etc/syslog-ng/syslog-ng.conf:5] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:5] Compiling d_mail reference [destination] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling d_mail sequence [destination] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed junction [log] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling s_sys reference [source] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling f_emergency reference [filter] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling f_emergency sequence [filter] at [/etc/syslog-ng/syslog-ng.conf:5] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:5] Compiling d_mlal reference [destination] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling d_mlal sequence [destination] at [/etc/syslog-ng/syslog-ng.conf:4] Compiling #unnamed junction [log] at [/etc/syslog-ng/syslog-ng.conf:4] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:4] Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling s_sys reference [source] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling f_news reference [filter] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling f_news sequence [filter] at [/etc/syslog-ng/syslog-ng.conf:5] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:5] Compiling d_spol reference [destination] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling d_spol sequence [destination] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed junction [log] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling s_sys reference [source] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling f_boot reference [filter] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling f_boot sequence [filter] at [/etc/syslog-ng/syslog-ng.conf:5] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:5] Compiling d_boot reference [destination] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling d_boot sequence [destination] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed junction [log] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling s_sys reference [source] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling f_cron reference [filter] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling f_cron sequence [filter] at [/etc/syslog-ng/syslog-ng.conf:5] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:5] Compiling d_cron reference [destination] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling d_cron sequence [destination] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed junction [log] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed sequence [log] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling s_net reference [source] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling s_net sequence [source] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed junction [log] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:3] Compiling d_log reference [destination] at [/etc/syslog-ng/syslog-ng.conf:6] Compiling d_log sequence [destination] at [/etc/syslog-ng/syslog-ng.conf:4] Compiling #unnamed junction [log] at [/etc/syslog-ng/syslog-ng.conf:4] Compiling #unnamed single [log] at [/etc/syslog-ng/syslog-ng.conf:4] Module loaded and initialized successfully; module='syslogformat' Module loaded and initialized successfully; module='linux-kmsg-format' Running application hooks; hook='1' Running application hooks; hook='3' syslog-ng starting up; version='3.5.6' If I comment out this line from syslog-ng.conf: # log { source(s_net); destination(d_log); }; Then restart syslog-ng (I know, I should use restart...) # systemctl start syslog-ng Starts no problem. # systemctl status syslog-ng.service ● syslog-ng.service - System Logger Daemon Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2018-10-12 12:00:47 EDT; 38s ago Docs: man:syslog-ng(8) Main PID: 5727 (syslog-ng) CGroup: /system.slice/syslog-ng.service └─5727 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid Oct 12 12:00:47 logger.com systemd[1]: Starting System Logger Daemon... Oct 12 12:00:47 logger.com systemd[1]: Started System Logger Daemon. It just won't stop (I guess by design) # systemctl stop syslog-ng Warning: Stopping syslog-ng.service, but it can still be activated by: syslog.socket # systemctl status syslog-ng.service ● syslog-ng.service - System Logger Daemon Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled) Active: active (running) since Fri 2018-10-12 12:01:55 EDT; 1s ago Docs: man:syslog-ng(8) Main PID: 5755 (syslog-ng) CGroup: /system.slice/syslog-ng.service └─5755 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid Oct 12 12:01:55 logger.com systemd[1]: Starting System Logger Daemon... Oct 12 12:01:55 logger.com systemd[1]: Started System Logger Daemon. But if I uncomment that line and restart: # systemctl restart syslog-ng Job for syslog-ng.service failed because the control process exited with error code. See "systemctl status syslog-ng.service" and "journalctl -xe" for details. # service syslog-ng status Redirecting to /bin/systemctl status syslog-ng.service ● syslog-ng.service - System Logger Daemon Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled) Active: failed (Result: start-limit) since Fri 2018-10-12 12:04:18 EDT; 2s ago Docs: man:syslog-ng(8) Process: 5811 ExecStart=/usr/sbin/syslog-ng -F -p /var/run/syslogd.pid (code=exited, status=2) Main PID: 5811 (code=exited, status=2) Status: "Starting up... (Fri Oct 12 12:04:18 2018" Oct 12 12:04:18 logger.com systemd[1]: Unit syslog-ng.service entered failed state. Oct 12 12:04:18 logger.com systemd[1]: syslog-ng.service failed. Oct 12 12:04:18 logger.com systemd[1]: syslog-ng.service holdoff time over, scheduling restart. Oct 12 12:04:18 logger.com systemd[1]: start request repeated too quickly for syslog-ng.service Oct 12 12:04:18 logger.com systemd[1]: Failed to start System Logger Daemon. Oct 12 12:04:18 logger.com systemd[1]: Unit syslog-ng.service entered failed state. Oct 12 12:04:18 logger.com systemd[1]: syslog-ng.service failed. Oct 12 12:04:18 logger.com systemd[1]: start request repeated too quickly for syslog-ng.service Oct 12 12:04:18 logger.com systemd[1]: Failed to start System Logger Daemon. Oct 12 12:04:18 logger.com systemd[1]: syslog-ng.service failed. It won't work. If I change syslog-ng.conf to a specific IP in the source I get nearly the same results. One note: Occasionally, in the journalctl -xe, I get the following: Oct 12 12:08:55 logger.com systemd[1]: Unit syslog-ng.service entered failed state. Oct 12 12:08:55 logger.com systemd[1]: syslog-ng.service failed. Oct 12 12:08:55 logger.com systemd-journal[494]: Forwarding to syslog missed 3 messages. -- Subject: One or more messages could not be forwarded to syslog -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- One or more messages could not be forwarded to the syslog service -- running side-by-side with journald. This usually indicates that the -- syslog implementation has not been able to keep up with the speed of -- messages queued. But that doesn't show up all the time. As far as options, it doesn't look like https://www.centos.org/forums/viewtopic.php?t=58296 https://www.syslog-ng.com/community/b/blog/posts/troubleshooting-java-suppo… rsyslog is disabled # systemctl status rsyslog ● rsyslog.service - System Logging Service Loaded: loaded (/usr/lib/systemd/system/rsyslog.service; disabled; vendor preset: enabled) Active: inactive (dead) Oct 11 19:03:17 logger.com systemd[1]: Starting System Logging Service... Oct 11 19:03:17 logger.com systemd[1]: Started System Logging Service. Oct 11 20:52:56 logger.com systemd[1]: Stopping System Logging Service... Oct 11 20:52:56 logger.com systemd[1]: Stopped System Logging Service. https://www.centos.org/forums/viewtopic.php?t=57379 Nothing seems to be blocking access to the log directory: # lsof /dev/log COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME systemd 1 root 42u unix 0xffff8800368952c0 0t0 9277 /dev/log systemd-j 494 root 5u unix 0xffff8800368952c0 0t0 9277 /dev/log https://www.reddit.com/r/linuxadmin/comments/5ezj5c/syslogng_service_doesnt… I've also looked at https://unix.stackexchange.com/questions/202044/syslog-ng-service-not-start… and https://lists.balabit.hu/pipermail/syslog-ng/2017-September/024024.html

1 0

RFC-5424 configuration with IPV6 is not working
by venkateswarlu vinjamuri 09 Oct '18

09 Oct '18

Hi, *issue*: syslog-ng is not starting if we configure IPV6 IP along with RFC-5424 format Using below command to run syslog-ng: /sbin/syslog-ng -f /etc/syslog-ng/syslog-ng.conf -R /var/syslog-ng.persist -p /var/syslog-ng.pid Getting the below error: ------------------------------ *syslog-ng: Error changing to Unknown syslog transport specified, please use one of udp, tcp, or tls; transport='udp6', id='df_remote_0#0'* Could anyone please let me know what should be the configuration in syslog-ng.conf for IPV6 syslog-ng server IP with RFC-5424 format. we are using syslog-ng-3.3.7 version. *If the below configuration is correct, will it work if we upgrade to newer version with the below configuration?* I am using the below configuration in syslog-ng.conf: ======================================== options { stats_freq (0); flush_lines (0); time_reopen (10); log_fifo_size (10000); chain_hostnames (off); use_dns (no); create_dirs (no); keep_hostname (no); perm(0640); group("root"); }; # sources source s_all { internal(); unix-stream("/dev/log"); file("/proc/kmsg" program_override("kernel: ")); }; filter f_remote { facility(local7); }; destination df_remote_0 {syslog("xxxx:xxxx:xxxx:xxxx:xxxx" transport("udp6"));}; log { source(s_all); filter(f_remote); destination(df_remote_0 Please help if there is any issue in the above configuration? Regards, V/

2 8

syslong_ng and bind9
by Ruben Safir 09 Oct '18

09 Oct '18

Hello I am trying to log named to syslog_ng and it is not set up by default on arch or artex. I set up named for logging, I think logging { category default { log_syslog; }; channel log_syslog { syslog; }; }; in the config file for syslog_ng I had added destination d_mail { file("/var/log/mail.log"); }; destination d_named { file("/var/log/named.log"); }; destination d_news { file("/var/log/news.log"); }; I am looking at filters and have no idea what to do filter f_auth { facility(auth); }; filter f_authpriv { facility(auth, authpriv); }; filter f_syslog { program(syslog-ng); }; filter f_cron { facility(cron); }; filter f_daemon { facility(daemon); }; filter f_kernel { facility(kern) and not filter(f_iptables); }; filter f_lpr { facility(lpr); }; filter f_mail { facility(mail); }; filter f_news { facility(news); }; filter f_user { facility(user); }; filter f_uucp { facility(uucp); }; filter f_ppp { facility(local2); }; filter f_debug { not facility(auth, authpriv, news, mail); }; filter f_messages { level(info..warn) and not facility(auth, authpriv, mail, news, cron) and not program(syslog-ng) and not filter(f_iptables); }; filter f_everything { level(debug..emerg) and not facility(auth, authpriv); }; filter f_emergency { level(emerg); }; filter f_info { level(info); }; filter f_notice { level(notice); }; filter f_warn { level(warn); }; filter f_crit { level(crit); }; filter f_err { level(err); }; filter f_iptables { match("IN=" value("MESSAGE")) and match("OUT=" value("MESSAGE")); }; filter f_acpid { program("acpid"); }; I don't know if there is a built in facilty for named. The docs for the server and very detailed and rich, and overwhelming. Ruben -- So many immigrant groups have swept through our town that Brooklyn, like Atlantis, reaches mythological proportions in the mind of the world - RI Safir 1998 http://www.mrbrklyn.com DRM is THEFT - We are the STAKEHOLDERS - RI Safir 2002 http://www.nylxs.com - Leadership Development in Free Software http://www.brooklyn-living.com Being so tracked is for FARM ANIMALS and and extermination camps, but incompatible with living as a free human being. -RI Safir 2013

2 2

'network' Destination With Hostname Resolution (IPv4 vs IPv6)
by David Hauck 06 Oct '18

06 Oct '18

Hi, I have a question regarding how to specify a network destination when using a hostname when the hostname can be resolved to either IPv4 or IPv6. In particular what should be specified by the ip-protocol() parameter? There are some configuration scenarios and/or target installations that don't know a priori whether the DNS configuration will resolve to an IPv4 or IPv6 address. E.g., destination d_tcp6 { network( "myhost" port(514) transport(udp) ip-protocol(6 or 4 or ??) ); }; It seems like it would simple enough to have syslog-ng simply validate the resulting IP address string to determine which of ip-protocol(4) or ip-protocol(6) is actually needed. In fact, I would argue that specifying an IP address string (as the "<destination-address>" value) could result in the same determination (the address string necessarily unambiguously determines whether the reference is an IPv4 or an IPv6 address and I would think there is a 1-1 relationship between this determination and whether ip-protocol(4) or ip-protocol(6) is used - (in other words it would never make sense to have these mixed: "::1" and ip-protocol(4) would be invalid). Thanks, -David

3 12

pattern usage and optimization
by Joel Carnat 01 Oct '18

01 Oct '18

Hi, I want to parse my sshd logs to store information in a remote database. I already did it using logstash. But I just discovered syslog-ng can do such things using patterndb. I could manage to setup a few <pattern> but I have difficulties building generic rules. I end up with 4 or 5 rules where I can only deal with one or two using logstash. So I expect to be missing something with patterns :) Here's a log example: Disconnected from user joe 192.168.0.5 port 50121 Disconnected from invalid user www 192.168.0.7 port 6794 [preauth] Disconnected from authenticating user root 192.168.0.3 port 52591 [preauth] So I wrote those three patterns:  <pattern>@ESTRING:EVENT: from @user @ESTRING:USERNAME: @@ESTRING:IP: @port @NUMBER:PORT:@</pattern>  <pattern>@ESTRING:EVENT: from @invalid user @ESTRING:USERNAME: @@ESTRING:IP: @port @NUMBER:PORT:@@ANYSTRING:EXTRA:@</pattern>  <pattern>@ESTRING:EVENT: from @authenticating user @ESTRING:USERNAME: @@ESTRING:IP: @port @NUMBER:PORT:@@ANYSTRING:EXTRA:@</pattern> To me, those 3 lines can be described using a single expression this way : ("Disconnected from") ("user"|"invalid user"|"authenticating user") (username) (ip_host) port (ip_port)(empty|extra_stuff) Basically, the features I couldn't find are : - "match a defined string and affect to variable" - "match a string or another and affect to variable" - "match a string or EOL and affect to variable if not empty". Is it possible to have a single pattern that would lead to have: - EVENT = "Disconnected from" - METHOD = "user" | "invalid user" | "authenticating user" - USERNAME = <parsed username> - IP = <parsed ip address> - PORT = <parsed port number> - EXTRA = <empty> | <parsed extra information> Thanks for you help.

2 3

Undesirable behavior from Cisco parser?
by Nik Ambrosch 27 Sep '18

27 Sep '18

Recently I upgraded my centralized loghost from 3.9 -> 3.15 and I noticed that some of my cisco devices started being logged in an undesirable format... I don't want to enable the cisco parser because more than just cisco messages get delivered to this interface. Here are the relevant fields that have changed before/after the upgrade: syslog-ng 3.9, before upgrade --- ${FULLHOST}: "mydevice.com" ${PROGRAM}: "" message: "%CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd IPSEC packet has invalid spi for..." syslog-ng 3.15, before upgrade --- ${FULLHOST}: ":" ${PROGRAM}: "%CRYPTO-4-RECVD_PKT_INV_SPI" ${MSG}: "decaps: rec'd IPSEC packet has invalid spi for..." Is this unintended behavior or a bug? This particular device is a Cisco 3845 running ios 12.4(22)T4. Thanks in advance.

5 17

disk-buffer in elasticsearch2 destination loses messages if docker container is killed
by Jose Angel Santiago 25 Sep '18

25 Sep '18

Hi, I'm running syslog-ng (with an elasticsearch2 destination configured) within a docker container, and I'm trying to avoid loss of messages if I kill the docker container and I start it again. This is my scenary: - A service which produces 20 lines of log per second - A sislog-ng instance reading from a wildcard-file source (but actually it only reads logs from the above service, let's call it syslog-agent), which sends all logs to another syslog-ng instance (the one running in a docker container, let's call it syslog-relay) though a network destination. - The syslog-relay sends messages to an elasticsearch instance, with following configuration: options { chain-hostnames(no); use-dns(no); keep-hostname(yes); owner("syslog-ng"); group("stratio"); perm(0640); time-reap(30); mark-freq(10); stats-freq(0); bad-hostname("^gconfd$"); flush-lines(100); log-fifo-size(1000); }; *destination d_elastic_default_0 { elasticsearch2( cluster("myelastic") cluster-url("https://myelastic.logs:9200 <https://myelastic.logs:9200>") client_mode("https") index("default") type("log") flush-limit(1) disk-buffer( mem-buf-size(16M) disk-buf-size(16M) reliable(yes) dir("/syslog-ng/log") ) http-auth-type("clientcert") java-keystore-filepath("/etc/syslog-ng/certificates/syslog-relay.jks") java-keystore-password("XXXXXX") java-truststore-filepath("/etc/syslog-ng/certificates/ca-bundle.jks") java-truststore-password("XXXXXXXXXX") );};* - The dir "/syslog-ng/log" is mapped to a path "/tmp/buffer" from the host where the docker container is running, so when I kill the docker container, the buffer file is not lost. - I've set flush-limit to 1 because I thought that I may lost 1 message only as much. This architecture is working fine (flush-limit=1 makes very slow, but for this test is ok), but if I kill the syslog-relay docker container, wait 5 to 10 seconds and start it again from scratch, I can see that several hundreds of logs are missing in elasticsearch. I check it by stopping the logger service and letting syslog-ng agent & relay to finish the process enqueued messages. I can see in the syslog-agent stats that all logs messages have been processed, so it seems the problem is on the syslog-relay. Is this behaviour expected? If so, how can I protect against loss of messages in case of a syslog-relay docker container unexpected kill? Thanks in advance. -- | Jose Angel Santiago [image: Logo_signature2.png] <http://www.stratio.com/> Vía de las dos Castillas, 33, Ática 4, 3ª Planta 28224 Pozuelo de Alarcón, Madrid, Spain +34 918 286 473 | www.stratio.com <https://twitter.com/stratiobd> <https://www.linkedin.com/company/stratiobd> <https://www.youtube.com/c/StratioBD>

3 16

Syslog-ng missing few logs to write to local file after showing in tcpdump
by Harish Shetty 24 Sep '18

24 Sep '18

Hi All I have syslog-ng (syslog-ng-3.9.1-1.el6.x86_64) which is running on Red Hat 6.6. I have rule defined as mentioned below destination d_networkx { file("/mnt/u001/syslog/$HOST/$YEAR$MONTH$DAY/network.log"); }; But syslog-ng is unable to write some logs to local file even though we see those logs in the tcpdump? What could be causing this. And one more thing need to know , is ther any way to check per filter (d_networkx) how many we received and how many we Processed(wrote) to a file Regards Harish Shetty

3 2

Insider 2018-09: SCL; Splunk .conf18; failover; human readable numbers; webinars;
by Czanik, Péter 20 Sep '18

20 Sep '18

Dear syslog-ng users, This is the 69th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Creating your first block for the syslog-ng configuration library (SCL) --------------------------------------------------------- The syslog-ng configuration library (SCL) is a collection of ready-to-use configuration snippets that hide away the complexity of the specifics of your log processing pipeline. If you already reuse parts of your configuration on different machines, it is not rocket science to turn these parts into blocks, making your configuration files a lot simpler. And if they are not site-specific, you can even help other syslog-ng users by submitting them to be included in SCL, and distributed in the official syslog-ng releases. https://www.syslog-ng.com/community/b/blog/posts/creating-your-first-block-… syslog-ng at Splunk .conf18 --------------------------- Going to Splunk .conf18? See a demo, meet with us (Booth #M49) and talk to syslog-ng creator, Balázs Scheidler. He will also give a talk on 02 October, 2:15 - 3:00 p.m. about simplifying complex log data ingestion with syslog-ng: https://conf.splunk.com/learn/session-catalog.html?search=ITS2133#/ Client-side failover and failback using syslog-ng ------------------------------------------------- When you have multiple syslog servers collecting logs, syslog-ng, on the client side, can fail over to secondary servers if the primary one becomes unavailable. It can also fail back to the primary server soon after it is back online – if configured so. Using client-side failover lowers the risk of message loss and makes the maintenance of servers easier. https://www.syslog-ng.com/community/b/blog/posts/client-side-failover-and-f… Human readable numbers in syslog-ng ----------------------------------- Writing an extra zero at the wrong place in a configuration file can result in a filled-up partition and data loss. This is where human readable numbers in syslog-ng can help: using kilo / mega / giga instead of writing many-digit numbers can save you lots of debugging and headache. https://www.syslog-ng.com/community/b/blog/posts/human-readable-numbers-in-… WEBINARS You can watch our past webinars: * Logging in docker with syslog-ng: https://www.brighttalk.com/webcast/16207/333091 * Supercharge your syslog-ng parsing: https://www.brighttalk.com/webcast/16207/330198 Or register for our upcoming syslog-ng webinars: * High performance log streaming to HDFS with syslog-ng: https://www.syslog-ng.com/event/high-performance-log-streaming-to-hdfs-with… Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> One Identity / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

Mutual TLS issues
by Matthew Pounsett 19 Sep '18

19 Sep '18

I'm trying to get mutual TLS validation working and seem to be having issues with the config. The situation I've got is that I'm trying to accept a stream from another organization. And so my preferred configuration involves two CA's.. mine and theirs. My syslog-ng server needs to validate their server certificate using their CA certificate, and their syslog-ng client needs to validate my server certificate using my CA certificate. I've set up a test using the server that will be production, and a handy VM I control. I assumed that it would be as simple as using the example configs from the manual [0], and anywhere a CA cert is involved just pick the appropriate one: store their CA on my server, and my CA on "their" server (the test VM). That didn't work. I've backed down to trying to get this working using all certificates signed by my CA, but I can't seem to make that work either. The error timing I'm getting doesn't quite match similar errors I can see reported online. If I don't have the correct CA cert on the client, then I see a connect, immediate disconnect, and a "SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca" error on the server. If I do have the CA cert on the client, I see a connect ... and then nothing until I try to send a message. Then I get the same error on the server and a logged "Certificate validation failed" error on the client referencing the server's cert, and a disconnect. Server logs: Sep 17 23:17:49 adm syslog-ng[31128]: Syslog connection accepted; fd='24', client='AF_INET(216.235.10.40:54063)', local='AF_INET(64.191.0.17:6514)' Sep 17 23:18:01 adm syslog-ng[31128]: SSL error while reading stream; tls_error='SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca' Sep 17 23:18:01 adm syslog-ng[31128]: I/O error occurred while reading; fd='24', error='Connection reset by peer (104)' Sep 17 23:18:01 adm syslog-ng[31128]: Syslog connection closed; fd='24', client='AF_INET(216.235.10.40:54063)', local='AF_INET(64.191.0.17:6514)' Client logs: Sep 17 23:17:48 ubuntu syslog-ng[4603]: syslog-ng starting up; version='3.13.2' Sep 17 23:17:48 ubuntu syslog-ng[4603]: Syslog connection established; fd='13', server='AF_INET(64.191.0.17:6514)', local='AF_INET(0.0.0.0:0)' Sep 17 23:18:00 ubuntu syslog-ng[4603]: Certificate validation failed; subject='emailAddress=xxx(a)dns-oarc.net, CN=64.191.0.17, O=OARC\, Inc., L=Indianapolis, ST=Indiana, C=US', issuer='emailAddress=xxx(a)dns-oarc.net, CN=OARC Inc. Certificate Authority, O=OARC\, Inc., ST=Indiana, C=US', error='unable to get local issuer certificate', depth='0' Sep 17 23:18:00 ubuntu syslog-ng[4603]: SSL error while writing stream; tls_error='SSL routines:tls_process_server_certificate:certificate verify failed' Sep 17 23:18:00 ubuntu syslog-ng[4603]: I/O error occurred while writing; fd='13', error='Broken pipe (32)' Sep 17 23:18:00 ubuntu syslog-ng[4603]: Syslog connection broken; fd='13', server='AF_INET(64.191.0.17:6514)', time_reopen='60' Note the 12 second gap between startup and cert validation failure. The failure coincides with logging a message that should be sent from the client to the server. Server config: source tls_source { network(ip(64.191.0.17) port(6514) transport("tls") tls( key-file("/etc/ssl/private/64.191.0.17.key") cert-file("/etc/ssl/certs/64.191.0.17.crt") ca-dir("/etc/ssl/certs") ) ); }; Client config: destination d_tls { network("64.191.0.17" port(6514) transport("tls") tls( ca-dir("/etc/ssl/certs") key-file("/etc/ssl/private/216.235.10.40.key") cert-file("/etc/ssl/certs/216.235.10.40.crt") ) ); }; I have the CA cert stored as /etc/ssl/certs/server_ca.pem on both systems. Any idea what's wrong here? And, should I be able to get this to work using different CAs for the client and server certs? Thanks [0]: <https://www.syslog-ng.com/technical-documents/doc/ syslog-ng-open-source-edition/3.16/mutual-authentication- using-tls#TOPIC-956368>

2 3

@ as delimiter in patterndb parser
by joao reis 13 Sep '18

13 Sep '18

How can I specify @ as a delimiter in patterndb? I want to parse user and domain separated in email adresses. Thanks in advance.

2 1

Packet loss.
by vijay amruth 13 Sep '18

13 Sep '18

Hello everyone, we’ve been experiencing packet loss on syslog-ng collectors (these are open stack VMs), lately we have increased ingestion and that refelcted packet loss ( especially UDP). Tuning parameters didn’t help. If anyone here on the list has experienced a similar issue please share your resolution or work around. We are at 3.9.1, we are planning to update to 3.17.x for better data handling/other bug fixes. As an alternate approach we have increased the VM capacity ( no of VMs). If there is any that I missed, please point. Any thoughts? Thank you, Vijay Amrut. -- Thanks, Vijay Amrut.

3 4

Crash on syslog-ng 3.8.1
by xen rkar 12 Sep '18

12 Sep '18

Hi, Looking to see if this is a know issue, cause i have been unsucessful in find an exact match of the signature anywhere in the forums. Regards, rkar #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 51 ../sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt #0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51 #1 0x76c5775c in __GI_abort () at abort.c:79 #2 0x76f537a8 in iv_fatal (fmt=0x76f750dc "iv_timer_unregister: timer index %d > %d") at ../../../../syslog-ng-3.8.1/lib/ivykis/src/iv_fatal.c:57 #3 0x76f53f4c in iv_timer_unregister (_t=0x17e5284) at ../../../../syslog-ng-3.8.1/lib/ivykis/src/iv_timer.c:305 #4 0x76f53fb8 in iv_run_timers (st=st@entry=0x17a50c0) at ../../../../syslog-ng-3.8.1/lib/ivykis/src/iv_timer.c:148 #5 0x76f55ad8 in iv_main () at ../../../../syslog-ng-3.8.1/lib/ivykis/src/iv_main_posix.c:107 #6 0x76f24524 in main_loop_run () at ../syslog-ng-3.8.1/lib/mainloop.c:497 #7 0x000113e0 in main (argc=70624, argv=0x7ea96054) at ../syslog-ng-3.8.1/syslog-ng/main.c:289

2 1

syslog-ng never frees RAM
by Jose Angel Santiago 10 Sep '18

10 Sep '18

Hi, I'm using syslog-ng 3.16.1 with wildcard-file as source (let's call it "syslog-agent), which sends log messages to another syslog-ng acting as a relay. I've noticed that syslog-agent instances RAM consumption keeps increasing until they leave no free memory in the cluster (each server has 64G of RAM). In my use case, new folders & files are created constantly under base-dir folder, but every 2 days obsolete folders & files are deleted. I assumed that syslog-ng would free some RAM every time those folders & files are deleted, but it doesn't, not even if I run a syslog-ng-ctl reload operation. log-fifo-size value is 345000, so I assume it can't be a buffer situation since 345000 messages can't occupy 40-50 GB of memory. I've performed the following test to reproduce the situation in small scale: - Launch a syslog-agent with a wildcard-file source reading from "/tmp/test/" base-dir. syslog-agent RAM usage is about 125M. - Run a simple script to create complex folder hierarchy under /tmp/test and some files with 5000 log messages to read from. - Wait until syslog-agent RAM usage gets 1GB - Stop script execution and wait until syslog-agent has send all logs to relay. - Delete everything under /tmp/test and execute syslog-ng-ctl reload operation - 24 hours after, syslog-agent RAM usage still is 1GB I've used heaptrack tool as a try to find a memory leak in syslog-ng, you can see in the attached image that iv_list_empty function in iv_list.h file is where most of the RAM usage is. How do I get syslog-ng to free RAM? Or is it a memory leak? Thanks in advance. -- | Jose Angel Santiago [image: Logo_signature2.png] <http://www.stratio.com/> Vía de las dos Castillas, 33, Ática 4, 3ª Planta 28224 Pozuelo de Alarcón, Madrid, Spain +34 918 286 473 | www.stratio.com <https://twitter.com/stratiobd> <https://www.linkedin.com/company/stratiobd> <https://www.youtube.com/c/StratioBD>

3 7

(no subject)
by R.Naveen Naidu 08 Sep '18

08 Sep '18

Hi, I need a help to choose the syslog-ng package for Linux. What are the things need to consider when choosing the latest syslog-ng to Linux? Actually, I wanted to cross-compile the syslog-ng and load into the customized Linux version *4.1.35rt41* The current latest syslog-ng version is* 3.17.2* from below site and planning to use this. https://github.com/balabit/syslog-ng/releases -- Thanks & Regards Naveen

3 8

problems compiling 3.17.2 on Ubuntu 16.0.4 with java
by Russell Fulton 04 Sep '18

04 Sep '18

I am trying to build syslog-ng with java and json support on Ubuntu 16.0.4. I can’t use prepackaged version because ubuntu have decided that one must not have both rsyslog (which is our corporate standard) and syslog-ng (which I need for its pattern matching in an application) on the same machine. Installing one package removes the other. As well as the standard build tools I have installed these packages via apt: jsoncpp, jre and gradle. Simply running ./configure ran through but did not find java (i.e. java and java-module were both set to no). I added —enable java and configure terminated complaining that it could not find java. I dug into the configure script and found that it was executing “which javac” and “javah” both of which failed. I changed JAVAC to “java” and the configure script then completed with both java and java module enabled. Running make then failed builder@secgendev02:~/src/syslog-ng-3.17.2$ make mkdir -p modules/java/syslog-ng-core/classes GEN modules/java/syslog-ng-core/classes/org/syslog_ng/LogMessage.class Unrecognized option: -d Error: Could not create the Java Virtual Machine. Error: A fatal exception has occurred. Program will exit. Makefile:20740: recipe for target 'modules/java/syslog-ng-core/classes/org/syslog_ng/LogMessage.class’ failed I am guessing that the version of java the jre package installed is not compatible with the configure script. builder@secgendev02:~/src/syslog-ng-3.17.2$ java -version openjdk version "1.8.0_181" OpenJDK Runtime Environment (build 1.8.0_181-8u181-b13-0ubuntu0.16.04.1-b13) OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode) I know that configure checks the java version so I am somewhat surprised… Any Ideas on what to try next? Russell

2 1