- syslog-ng - lists.balabit.hu

syslog-ng-3.25.1
by Laszlo Budai 12 Dec '19

12 Dec '19

3.25.1 Highlights * http-destination: Users now can specify the action for any HTTP result code. Use with response-action(response_code => action) in your http block. Available actions are: success, retry, drop and disconnect. (#3007<https://github.com/syslog-ng/syslog-ng/pull/3007>) * syslog-ng-cfg-db: Added a new script, which can provide the options of sources and destinations queried by the user. This tool can make the configuration of syslog-ng a lot easier. Use with ./syslog-ng-cfg-db.py from the contrib/config_database dir.(#2997<https://github.com/syslog-ng/syslog-ng/pull/2997>) * redis-destination: Improved the performance by 2 orders of magnitude. In our labor environment, now it operates at 25k EPS. (#2972<https://github.com/syslog-ng/syslog-ng/pull/2972>) Features * create-dirs(): Added to pipe() source/destination, and standardize the behavior. (#3018<https://github.com/syslog-ng/syslog-ng/pull/3018>, #2635<https://github.com/syslog-ng/syslog-ng/issues/2635>) * default-network-drivers: Added max-connections() option, to change the limit from 10. (#2961<https://github.com/syslog-ng/syslog-ng/pull/2961>) * checkpoint: Added support for timezone value at the end of timestamps. (#3033<https://github.com/syslog-ng/syslog-ng/pull/3033>) * filter/rewrite: Added disable-jit flag to disable JIT PCRE compilation. (#2992<https://github.com/syslog-ng/syslog-ng/pull/2992>, #2986<https://github.com/syslog-ng/syslog-ng/issues/2986>) * syslog-ng-ctl: Added export-config-graph option to visualize config graph. (#2990<https://github.com/syslog-ng/syslog-ng/pull/2990>) * build/travis: Added ARM64 arch support. (#2967<https://github.com/syslog-ng/syslog-ng/pull/2967>) * build/dbld: Readded CentOS 6 support. (#2860<https://github.com/syslog-ng/syslog-ng/pull/2860>, #2971<https://github.com/syslog-ng/syslog-ng/pull/2971>, #3028<https://github.com/syslog-ng/syslog-ng/pull/3028>) * python: Added Python 3.8 support. (#3017<https://github.com/syslog-ng/syslog-ng/pull/3017>) Bugfixes * tls: Fixed an infinite loop which occured, when a TLS connection broke. (#3026<https://github.com/syslog-ng/syslog-ng/pull/3026>, #3009<https://github.com/syslog-ng/syslog-ng/issues/3009>) * log-block: Fixed an issue, where inline network destinations disjointed the rest of the config. (#2989<https://github.com/syslog-ng/syslog-ng/pull/2989>, #2820<https://github.com/syslog-ng/syslog-ng/issues/2820>) * kafka/network-load-balancer: Fixed a crash when an argument was set to empty. (#3002<https://github.com/syslog-ng/syslog-ng/pull/3002>) * python-source: Fixed a memory corruption during reload. (#3014<https://github.com/syslog-ng/syslog-ng/pull/3014>) * python-destination: Actually use return value of open method. (#2998<https://github.com/syslog-ng/syslog-ng/pull/2998>, #2513<https://github.com/syslog-ng/syslog-ng/issues/2513>) * python-fetcher: Fixed FETCH_NO_DATA and FETCH_TRY_AGAIN constants. (#3012<https://github.com/syslog-ng/syslog-ng/pull/3012>) * python: Fixed python Exception reporting when no Exception happened. (#2995<https://github.com/syslog-ng/syslog-ng/pull/2995>) * telegram: Fixed the syntax error of the use-system-cert-store() option. (#2977<https://github.com/syslog-ng/syslog-ng/pull/2977>) * config: Throw error to single dots, which were ignored before. (#3000<https://github.com/syslog-ng/syslog-ng/pull/3000>) * file-destination: Delay ACKs until messages are written to disk. This fixes message drop on I/O error and message lost in the LogProtoFileWriter in case of a crash, by retrying to send the message. (#2985<https://github.com/syslog-ng/syslog-ng/pull/2985>) * http-destination: Handle global template options values. (#3020<https://github.com/syslog-ng/syslog-ng/pull/3020>) * timeutils: Fixed month and day name parsing, when only the first 2 characters matched. (#3035<https://github.com/syslog-ng/syslog-ng/pull/3035>) * logmsg: Added default PRI value (LOG_USER | LOG_NOTICE) to log messages created without initial parsing. (#2974<https://github.com/syslog-ng/syslog-ng/pull/2974>) * packaging: Added ordering dependencies network.target and network-online.target to the service files. (#2994<https://github.com/syslog-ng/syslog-ng/pull/2994>, #2667<https://github.com/syslog-ng/syslog-ng/issues/2667>) * amqp: Support older (0.7.1) version (#2999<https://github.com/syslog-ng/syslog-ng/pull/2999>) * loggen: Set plugin path in installation time. (#3019<https://github.com/syslog-ng/syslog-ng/pull/3019>) * timeutils/patterndb: Fixed some undefined behaviours. (#2969<https://github.com/syslog-ng/syslog-ng/pull/2969>) * stomp: Fixed a buffer over-read on connection. (#2988<https://github.com/syslog-ng/syslog-ng/pull/2988>) * pseudofile: Fixed a crash, when template() option is not set. (#2988<https://github.com/syslog-ng/syslog-ng/pull/2988>) * wildcard-source: Fixed a crash, when max-files() was set to 0. (#2988<https://github.com/syslog-ng/syslog-ng/pull/2988>) Other changes * syslog-ng-debun: Various maintenance updates and small fixes. (#2993<https://github.com/syslog-ng/syslog-ng/pull/2993>) * scl: Avoid @requires loading the plugins themselves. (#2887<https://github.com/syslog-ng/syslog-ng/pull/2887>) Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Andras Mitzki, Antal Nemes, Attila Szakacs, Balazs Scheidler, Clément Besnier, Gabor Nagy, jadhavsumit98, Janos Szigetvari, Laszlo Budai, Laszlo Szemere, László Várady, MikeLim, Nikita Uvarov, Norbert Takacs, pabloli, Péter Kókai, Zoltan Pallagi.

1 0

Insider 2019-12: Kibana 7 & GeoIP; PE 6 to 7 upgrade; RHEL 8; Elastic stack;
by Peter Czanik (pczanik) 12 Dec '19

12 Dec '19

Dear syslog-ng users, This is the 77th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Set up Kibana 7 for syslog-ng & GeoIP ------------------------------------- Version 7 of the Elastic stack was released a few months ago, and brought several breaking changes that affect syslog-ng. In my previous blog post, I gave details about how it affects sending GeoIP information to Elasticsearch. From this blog post you can learn about the Kibana side, which has also changed considerably compared to previous releases. Configuration files for syslog-ng are included, but not explained in depth, as that was already done in previous posts. https://www.syslog-ng.com/community/b/blog/posts/set-up-kibana-7-for-syslog… Upgrading a syslog-ng PE 6 configuration to 7 --------------------------------------------- Upgrading syslog-ng PE 6 to 7 is no more difficult than any major version upgrade. The key here is to use the latest version of PE 7 (at least version 7.0.17), as it has the necessary compatibility fixes to make the upgrade experience smoother. From this blog, you can learn how to upgrade a simple configuration containing a wildcard file source and an RLTP source to work correctly with syslog-ng PE 7. https://www.syslog-ng.com/community/b/blog/posts/upgrading-a-syslog-ng-pe-6… State of syslog-ng on RHEL 8 / CentOS 8 --------------------------------------- Version 8, a new major version of Red Hat Enterprise Linux was released this spring. Now that CentOS 8 is also available, there is a rapidly growing interest in syslog-ng running on these platforms. From this blog, you can learn about the availability of the current syslog-ng package and future plans. https://www.syslog-ng.com/community/b/blog/posts/state-of-syslog-ng-on-rhel… Using syslog-ng with the Elastic stack -------------------------------------- One of the most popular destinations of syslog-ng is Elasticsearch. Any time a new language binding was introduced to syslog-ng, someone implemented an Elasticsearch destination for it. For many years, the official Elasticsearch destination for syslog-ng was implemented in Java. With the recent enhancements to the http() destination of syslog-ng, a new, native C-based implementation called the elasticsearch-http() destination is available. https://www.syslog-ng.com/community/b/blog/posts/using-syslog-ng-with-the-e… WEBINARS * You can achieve 1m+ UDP messages per second ingestion challenge: https://www.syslog-ng.com/webcast-ondemand/you-can-achieve-1m-udp-messages-… * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

How to syslog-ng execute shell script
by William Luiz Ribeiro Vasconcelos Da Silva 10 Dec '19

10 Dec '19

Hello People, How to configure the syslog-ng.conf file, to call a shell script, for example to do zip file. It is possible to do this? Tks, William Luiz Esta mensagem, incluindo seus anexos, pode conter informações privilegiadas e/ou de caráter confidencial, não podendo ser retransmitida sem autorização do remetente. Se você não é o destinatário ou pessoa autorizada para recebê-la, informamos que o seu uso, divulgação, cópia ou arquivamento são proibidos. Portanto, se você recebeu esta mensagem por engano, por favor nos informe respondendo imediatamente a este e-mail e delete o seu conteúdo. This message, including its attachments, may contain privileged or confidential information, and it must not be fowarded without the express authorization of the sender. If you are not the intended recipient, we hereby inform you that the use, disclosure, copy or filing are forbidden. So, if you received this message as a mistake, please inform us by answering this e-mail and deleting its contents Questo messaggio, inclusi gli allegati, potrebbe contenere informazioni privilegiate e/o riservate, e non deve essere ritrasmesse senza l'autorizzazione del mittente. Se non siete il destinatario o la persona autorizzata a riceverlo, informiamo che il suo utilizzo, diffusione, copia o archiviazione sono proibite. Quindi, se avete ricevuto questo messaggio per errore, per cortesia ci informi rispondendo immediatamente a questa email e cancelli il suo contenuto

2 5

Support for netflow logs
by Raghunath Adhyapak 04 Dec '19

04 Dec '19

Hi, I was trying to receive Netflow logs from firewall devices in syslog-ng and then forward to a central server. Does syslog-ng support netflow such that I can validate and filter out all non-netflow log lines? I also dumped some netflow logs to a file and found it to be binary. Therefore I haven't been able to ascertain the format and filtering mechanism. Any pointers on this topic would be helpful. Thanks Raghu

4 7

Its possible compress file using syslog-ng
by William Luiz Ribeiro Vasconcelos Da Silva 03 Dec '19

03 Dec '19

How to configure the syslog-ng.conf to do compress file? Its possible to use this feature inside of file structure: file("/<dir>/${HOST}_${YEAR}${MONTH}${DAY}${HOUR}${MIN}${SEC}_GMT${TZ}_${UNIQID}.log" template("${PROGRAM};${MESSAGE};\n") owner(user) group(user) perm (0755) Tks, Esta mensagem, incluindo seus anexos, pode conter informações privilegiadas e/ou de caráter confidencial, não podendo ser retransmitida sem autorização do remetente. Se você não é o destinatário ou pessoa autorizada para recebê-la, informamos que o seu uso, divulgação, cópia ou arquivamento são proibidos. Portanto, se você recebeu esta mensagem por engano, por favor nos informe respondendo imediatamente a este e-mail e delete o seu conteúdo. This message, including its attachments, may contain privileged or confidential information, and it must not be fowarded without the express authorization of the sender. If you are not the intended recipient, we hereby inform you that the use, disclosure, copy or filing are forbidden. So, if you received this message as a mistake, please inform us by answering this e-mail and deleting its contents Questo messaggio, inclusi gli allegati, potrebbe contenere informazioni privilegiate e/o riservate, e non deve essere ritrasmesse senza l'autorizzazione del mittente. Se non siete il destinatario o la persona autorizzata a riceverlo, informiamo che il suo utilizzo, diffusione, copia o archiviazione sono proibite. Quindi, se avete ricevuto questo messaggio per errore, per cortesia ci informi rispondendo immediatamente a questa email e cancelli il suo contenuto

1 0

Structure data set to "-"
by Debjyoti Mukherjee 03 Dec '19

03 Dec '19

Hello Trying to send logs to remote syslog server in RFC 5424 format. The STRUCTURE_DATA should be set to "-". What is the way to the this value to "-" Currently it is coming as [meta sequenceId="21"]. I am using Openwrt and the syslog version is 3.24 Thank you

2 5

Uniqid
by William Luiz Ribeiro Vasconcelos Da Silva 03 Dec '19

03 Dec '19

Hello People, I'm using the uniqid in my syslog-ng.conf , but when using it in the file name, the output was presented in hexadecimal, example: "8e99d7b2(a)000000000000040b.log" How to convert the output to decimal because we have a balancer layer to collect files ending in even numbers and another layer for odd numbers. This is part of syslog-ng.conf: "${HOST}_${YEAR}${MONTH}${DAY}${HOUR}${MIN}${SEC}_GMT${TZ}_${UNIQID}.log" Tks Esta mensagem, incluindo seus anexos, pode conter informações privilegiadas e/ou de caráter confidencial, não podendo ser retransmitida sem autorização do remetente. Se você não é o destinatário ou pessoa autorizada para recebê-la, informamos que o seu uso, divulgação, cópia ou arquivamento são proibidos. Portanto, se você recebeu esta mensagem por engano, por favor nos informe respondendo imediatamente a este e-mail e delete o seu conteúdo. This message, including its attachments, may contain privileged or confidential information, and it must not be fowarded without the express authorization of the sender. If you are not the intended recipient, we hereby inform you that the use, disclosure, copy or filing are forbidden. So, if you received this message as a mistake, please inform us by answering this e-mail and deleting its contents Questo messaggio, inclusi gli allegati, potrebbe contenere informazioni privilegiate e/o riservate, e non deve essere ritrasmesse senza l'autorizzazione del mittente. Se non siete il destinatario o la persona autorizzata a riceverlo, informiamo che il suo utilizzo, diffusione, copia o archiviazione sono proibite. Quindi, se avete ricevuto questo messaggio per errore, per cortesia ci informi rispondendo immediatamente a questa email e cancelli il suo contenuto

2 1

SYSLOGHOST is being replaced with IP
by Raghunath Adhyapak 29 Nov '19

29 Nov '19

Hi, I'm observing that syslog-ng is modifying the SYSLOGHOST in the incoming log line and outputting an IP instead. I would like to retain the incoming hostname in incoming syslog and forward the same information. Here is my incoming log line: <13>Nov 29 04:07:40 BVRM-DC04 AgentDevice=WindowsLog\tAgentLogFile=Security\tPluginVersion=7.2.8.91\tSource=Microsoft-Windows-Security-Auditing\tComputer= BVRM-DC04.xxxxxxxx.com\tOriginatingComputer=172.26.1.60\tUser=\tDomain=\tEventID=4634\tEventIDCode=4634\tEventType=8\tEventCategory=12545\tRecordNumber=166757582\tTimeGenerated=1575029259\tTimeWritten=1575029259\tLevel=Log Always\tKeywords=Audit Success\tTask=SE_ADT_LOGON_LOGOFF\tOpcode=Info\tMessage=An account was logged off. Outgoing log line: <13>Nov 29 04:07:40 172.22.2.55 AgentDevice=WindowsLog\tAgentLogFile=Security\tPluginVersion=7.2.8.91\tSource=Microsoft-Windows-Security-Auditing\tComputer= BVRM-DC04.xxxxxxxx.com\tOriginatingComputer=172.26.1.60\tUser=\tDomain=\tEventID=4634\tEventIDCode=4634\tEventType=8\tEventCategory=12545\tRecordNumber=166757582\tTimeGenerated=1575029259\tTimeWritten=1575029259\tLevel=Log Always\tKeywords=Audit Success\tTask=SE_ADT_LOGON_LOGOFF\tOpcode=Info\tMessage=An account was logged off. FYI, this is log from Windows, but same is happening for syslog from other firewalls as well. My syslog-ng.conf: @version: 3.24 @include "scl.conf" ######################## # Sources ######################## source s_test_net { syslog(transport(udp) port(2514) ); }; ######################## # Destinations ######################## destination d_test { file("/tmp/test.log"); }; ######################## # Log paths ######################## log { source(s_test_net); destination(d_test); }; Thanks Raghu

2 2

syslog-ng packages for RHEL 8 / CentOS 8
by Peter Czanik (pczanik) 27 Nov '19

27 Nov '19

Hi, Today I reached a new milestone with syslog-ng packaging for RHEL 8 / CentOS 8, so here is a quick update: * syslog-ng 3.23 is available in EPEL 8 ( https://fedoraproject.org/wiki/EPEL ). It is missing the C-based Kafka driver (librdkafka in RHEL is too old) and the Java-based destination drivers (Kafka, Elastic, HDFS) due to missing dependencies in RHEL/EPEL. * highly experimental syslog-ng package missing most of the features needing extra dependencies. Core features, Elasticsearch (HTTP) destination are available in the package. You need to first enable the EPEL 8 repository and then add: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng324el8/ * Today I successfully added RHEL 8 support to my git snapshot repository: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng-githead/ Some dependencies, like recent version of Kafka are available from this repository, the rest from EPEL. As far as I am aware, all OSE features are now supported, except for tcp-wrapper, which was removed from Fedora & RHEL 8. This repository is regularly updated from git. * Once syslog-ng 3.25 is available, the upcoming syslog-ng325 repo will already support RHEL 8 / CentOS 8 and soon after my syslog-ng-stable repo will also receive RHEL 8 support. As usual, any feedback is very welcome and will be handled with best effort. I'd especially welcome feedback on my syslog-ng-githead repo -- even if that's not intended for production environments -- as that's where I prepare for the next syslog-ng release. Bye, Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

syntax error when using syslog-ng.conf from debian packaging
by Alexandre Santos 21 Nov '19

21 Nov '19

Hi, I am running into the following error when running the syslog-ng with the syslog-ng.conf from debian packaging: *Error parsing affile, syntax error, unexpected ')', expecting LL_IDENTIFIER or LL_STRING in /usr/local/etc/syslog-ng.conf:68:34-68:35:63 #64 destination d_console { usertty("root"); };6566 # Virtual console.67 #68----> destination d_console_all { file(`tty10`); };68----> ^* Do you have any idea about this? Thanks in advance, Alex

3 4