December 2019 - syslog-ng - lists.balabit.hu

3.25 and my RPM stable repos
by Peter Czanik (pczanik) 13 Jan '20

13 Jan '20

Hi, Version 3.25 of syslog-ng was released a week ago, and it seems to be rock solid. Still, as today is my last working day this year, I feel more comfortable to push it into my stable repos only on the first week of January, when I am back. If you still want to experiment with 3.25 on openSUSE/SLES or fedora/RHEL/CentOS you can do that in the versioned repositories. Note that RHEL 8 / CentOS 8 support is now merged with the rest, it's not in a separate repository any more. suse: https://build.opensuse.org/project/show/home:czanik:syslog-ng325 red hat: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng325/ Bye, Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 1

State of syslog-ng on Mac
by Peter Czanik (pczanik) 20 Dec '19

20 Dec '19

Hi, I just published a blog about the state of syslog-ng on Mac: https://www.syslog-ng.com/community/b/blog/posts/state-of-syslog-ng-on-mac As I write about future plans at the end of the blog, you can see that I'm an almost complete newbie on the Mac (the last time I used it actively was on the Motorola 68040) and it's mostly a free time project. Still, any questions, suggestion or help are very welcome! Bye, Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

3 3

Add prefix Or suffix inside of FileName
by William Luiz Ribeiro Vasconcelos Da Silva 17 Dec '19

17 Dec '19

Hello everyone, It is possible during syslog-ng processing to insert a suffix, as we have a system that will collect these files. And if we keep the same suffix, we have a big problem inside of prodction enviroment. Here's an example of how the file generation part: destination d_model { file("/syslog_out/udp_files/${HOST}_${YEAR}${MONTH}${DAY}${HOUR}${MIN}_GMT${TZ}.log" template("${PROGRAM};${MESSAGE};\n") owner(syslog) group(syslog) perm (0755) ); }; Esta mensagem, incluindo seus anexos, pode conter informações privilegiadas e/ou de caráter confidencial, não podendo ser retransmitida sem autorização do remetente. Se você não é o destinatário ou pessoa autorizada para recebê-la, informamos que o seu uso, divulgação, cópia ou arquivamento são proibidos. Portanto, se você recebeu esta mensagem por engano, por favor nos informe respondendo imediatamente a este e-mail e delete o seu conteúdo. This message, including its attachments, may contain privileged or confidential information, and it must not be fowarded without the express authorization of the sender. If you are not the intended recipient, we hereby inform you that the use, disclosure, copy or filing are forbidden. So, if you received this message as a mistake, please inform us by answering this e-mail and deleting its contents Questo messaggio, inclusi gli allegati, potrebbe contenere informazioni privilegiate e/o riservate, e non deve essere ritrasmesse senza l'autorizzazione del mittente. Se non siete il destinatario o la persona autorizzata a riceverlo, informiamo che il suo utilizzo, diffusione, copia o archiviazione sono proibite. Quindi, se avete ricevuto questo messaggio per errore, per cortesia ci informi rispondendo immediatamente a questa email e cancelli il suo contenuto

2 2

json destination config help
by Zhang, Husen 12 Dec '19

12 Dec '19

Hi community, I'm trying to have syslog-ng to write logs to json. My d_json_syslog-ng.conf: [cid:image001.png@01D5B023.F072D8A0] ├── conf.d │ ├── d_json_syslog-ng.conf │ └── es.conf.bak ├── patterndb.d ├── scl.conf └── syslog-ng.conf

3 5

syslog-ng 3.25 packages for Debian/Ubuntu
by Laszlo Budai 12 Dec '19

12 Dec '19

Hi, syslog-ng 3.25.1[1] packages are available in OBS repositories[2, 3]. List of supported OSs: * Debian 8.0 * Debian 9.0 * Debian 10.0 * Raspbian 9.0 * Raspbian 10.0 * Ubuntu 14.04 * Ubuntu 16.04 * Ubuntu 18.04 * Ubuntu 18.10 * Ubuntu 19.04 * Ubuntu 19.10 Note that Debian 10 and Ubuntu 19.04 are available from a separate repository[3] and there is no java support on these distributions. The reason is that these OBS builders don't have the required openjdk packages. Install ------- example: Debian 9.0 1. get release key wget -qO - http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… | sudo apt-key add - 2. add repo to APT sources eg.: /etc/apt/sources.list.d/syslog-ng-obs.list deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… ./ Then `apt-get update` and `apt-get install syslog-ng-core` Links -------- [1] https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.25.1 [2] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… [3] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… regards, Laszlo Budai

1 0

syslog-ng-3.25.1
by Laszlo Budai 12 Dec '19

12 Dec '19

3.25.1 Highlights * http-destination: Users now can specify the action for any HTTP result code. Use with response-action(response_code => action) in your http block. Available actions are: success, retry, drop and disconnect. (#3007<https://github.com/syslog-ng/syslog-ng/pull/3007>) * syslog-ng-cfg-db: Added a new script, which can provide the options of sources and destinations queried by the user. This tool can make the configuration of syslog-ng a lot easier. Use with ./syslog-ng-cfg-db.py from the contrib/config_database dir.(#2997<https://github.com/syslog-ng/syslog-ng/pull/2997>) * redis-destination: Improved the performance by 2 orders of magnitude. In our labor environment, now it operates at 25k EPS. (#2972<https://github.com/syslog-ng/syslog-ng/pull/2972>) Features * create-dirs(): Added to pipe() source/destination, and standardize the behavior. (#3018<https://github.com/syslog-ng/syslog-ng/pull/3018>, #2635<https://github.com/syslog-ng/syslog-ng/issues/2635>) * default-network-drivers: Added max-connections() option, to change the limit from 10. (#2961<https://github.com/syslog-ng/syslog-ng/pull/2961>) * checkpoint: Added support for timezone value at the end of timestamps. (#3033<https://github.com/syslog-ng/syslog-ng/pull/3033>) * filter/rewrite: Added disable-jit flag to disable JIT PCRE compilation. (#2992<https://github.com/syslog-ng/syslog-ng/pull/2992>, #2986<https://github.com/syslog-ng/syslog-ng/issues/2986>) * syslog-ng-ctl: Added export-config-graph option to visualize config graph. (#2990<https://github.com/syslog-ng/syslog-ng/pull/2990>) * build/travis: Added ARM64 arch support. (#2967<https://github.com/syslog-ng/syslog-ng/pull/2967>) * build/dbld: Readded CentOS 6 support. (#2860<https://github.com/syslog-ng/syslog-ng/pull/2860>, #2971<https://github.com/syslog-ng/syslog-ng/pull/2971>, #3028<https://github.com/syslog-ng/syslog-ng/pull/3028>) * python: Added Python 3.8 support. (#3017<https://github.com/syslog-ng/syslog-ng/pull/3017>) Bugfixes * tls: Fixed an infinite loop which occured, when a TLS connection broke. (#3026<https://github.com/syslog-ng/syslog-ng/pull/3026>, #3009<https://github.com/syslog-ng/syslog-ng/issues/3009>) * log-block: Fixed an issue, where inline network destinations disjointed the rest of the config. (#2989<https://github.com/syslog-ng/syslog-ng/pull/2989>, #2820<https://github.com/syslog-ng/syslog-ng/issues/2820>) * kafka/network-load-balancer: Fixed a crash when an argument was set to empty. (#3002<https://github.com/syslog-ng/syslog-ng/pull/3002>) * python-source: Fixed a memory corruption during reload. (#3014<https://github.com/syslog-ng/syslog-ng/pull/3014>) * python-destination: Actually use return value of open method. (#2998<https://github.com/syslog-ng/syslog-ng/pull/2998>, #2513<https://github.com/syslog-ng/syslog-ng/issues/2513>) * python-fetcher: Fixed FETCH_NO_DATA and FETCH_TRY_AGAIN constants. (#3012<https://github.com/syslog-ng/syslog-ng/pull/3012>) * python: Fixed python Exception reporting when no Exception happened. (#2995<https://github.com/syslog-ng/syslog-ng/pull/2995>) * telegram: Fixed the syntax error of the use-system-cert-store() option. (#2977<https://github.com/syslog-ng/syslog-ng/pull/2977>) * config: Throw error to single dots, which were ignored before. (#3000<https://github.com/syslog-ng/syslog-ng/pull/3000>) * file-destination: Delay ACKs until messages are written to disk. This fixes message drop on I/O error and message lost in the LogProtoFileWriter in case of a crash, by retrying to send the message. (#2985<https://github.com/syslog-ng/syslog-ng/pull/2985>) * http-destination: Handle global template options values. (#3020<https://github.com/syslog-ng/syslog-ng/pull/3020>) * timeutils: Fixed month and day name parsing, when only the first 2 characters matched. (#3035<https://github.com/syslog-ng/syslog-ng/pull/3035>) * logmsg: Added default PRI value (LOG_USER | LOG_NOTICE) to log messages created without initial parsing. (#2974<https://github.com/syslog-ng/syslog-ng/pull/2974>) * packaging: Added ordering dependencies network.target and network-online.target to the service files. (#2994<https://github.com/syslog-ng/syslog-ng/pull/2994>, #2667<https://github.com/syslog-ng/syslog-ng/issues/2667>) * amqp: Support older (0.7.1) version (#2999<https://github.com/syslog-ng/syslog-ng/pull/2999>) * loggen: Set plugin path in installation time. (#3019<https://github.com/syslog-ng/syslog-ng/pull/3019>) * timeutils/patterndb: Fixed some undefined behaviours. (#2969<https://github.com/syslog-ng/syslog-ng/pull/2969>) * stomp: Fixed a buffer over-read on connection. (#2988<https://github.com/syslog-ng/syslog-ng/pull/2988>) * pseudofile: Fixed a crash, when template() option is not set. (#2988<https://github.com/syslog-ng/syslog-ng/pull/2988>) * wildcard-source: Fixed a crash, when max-files() was set to 0. (#2988<https://github.com/syslog-ng/syslog-ng/pull/2988>) Other changes * syslog-ng-debun: Various maintenance updates and small fixes. (#2993<https://github.com/syslog-ng/syslog-ng/pull/2993>) * scl: Avoid @requires loading the plugins themselves. (#2887<https://github.com/syslog-ng/syslog-ng/pull/2887>) Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Andras Mitzki, Antal Nemes, Attila Szakacs, Balazs Scheidler, Clément Besnier, Gabor Nagy, jadhavsumit98, Janos Szigetvari, Laszlo Budai, Laszlo Szemere, László Várady, MikeLim, Nikita Uvarov, Norbert Takacs, pabloli, Péter Kókai, Zoltan Pallagi.

1 0

Insider 2019-12: Kibana 7 & GeoIP; PE 6 to 7 upgrade; RHEL 8; Elastic stack;
by Peter Czanik (pczanik) 12 Dec '19

12 Dec '19

Dear syslog-ng users, This is the 77th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Set up Kibana 7 for syslog-ng & GeoIP ------------------------------------- Version 7 of the Elastic stack was released a few months ago, and brought several breaking changes that affect syslog-ng. In my previous blog post, I gave details about how it affects sending GeoIP information to Elasticsearch. From this blog post you can learn about the Kibana side, which has also changed considerably compared to previous releases. Configuration files for syslog-ng are included, but not explained in depth, as that was already done in previous posts. https://www.syslog-ng.com/community/b/blog/posts/set-up-kibana-7-for-syslog… Upgrading a syslog-ng PE 6 configuration to 7 --------------------------------------------- Upgrading syslog-ng PE 6 to 7 is no more difficult than any major version upgrade. The key here is to use the latest version of PE 7 (at least version 7.0.17), as it has the necessary compatibility fixes to make the upgrade experience smoother. From this blog, you can learn how to upgrade a simple configuration containing a wildcard file source and an RLTP source to work correctly with syslog-ng PE 7. https://www.syslog-ng.com/community/b/blog/posts/upgrading-a-syslog-ng-pe-6… State of syslog-ng on RHEL 8 / CentOS 8 --------------------------------------- Version 8, a new major version of Red Hat Enterprise Linux was released this spring. Now that CentOS 8 is also available, there is a rapidly growing interest in syslog-ng running on these platforms. From this blog, you can learn about the availability of the current syslog-ng package and future plans. https://www.syslog-ng.com/community/b/blog/posts/state-of-syslog-ng-on-rhel… Using syslog-ng with the Elastic stack -------------------------------------- One of the most popular destinations of syslog-ng is Elasticsearch. Any time a new language binding was introduced to syslog-ng, someone implemented an Elasticsearch destination for it. For many years, the official Elasticsearch destination for syslog-ng was implemented in Java. With the recent enhancements to the http() destination of syslog-ng, a new, native C-based implementation called the elasticsearch-http() destination is available. https://www.syslog-ng.com/community/b/blog/posts/using-syslog-ng-with-the-e… WEBINARS * You can achieve 1m+ UDP messages per second ingestion challenge: https://www.syslog-ng.com/webcast-ondemand/you-can-achieve-1m-udp-messages-… * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

How to syslog-ng execute shell script
by William Luiz Ribeiro Vasconcelos Da Silva 10 Dec '19

10 Dec '19

Hello People, How to configure the syslog-ng.conf file, to call a shell script, for example to do zip file. It is possible to do this? Tks, William Luiz Esta mensagem, incluindo seus anexos, pode conter informações privilegiadas e/ou de caráter confidencial, não podendo ser retransmitida sem autorização do remetente. Se você não é o destinatário ou pessoa autorizada para recebê-la, informamos que o seu uso, divulgação, cópia ou arquivamento são proibidos. Portanto, se você recebeu esta mensagem por engano, por favor nos informe respondendo imediatamente a este e-mail e delete o seu conteúdo. This message, including its attachments, may contain privileged or confidential information, and it must not be fowarded without the express authorization of the sender. If you are not the intended recipient, we hereby inform you that the use, disclosure, copy or filing are forbidden. So, if you received this message as a mistake, please inform us by answering this e-mail and deleting its contents Questo messaggio, inclusi gli allegati, potrebbe contenere informazioni privilegiate e/o riservate, e non deve essere ritrasmesse senza l'autorizzazione del mittente. Se non siete il destinatario o la persona autorizzata a riceverlo, informiamo che il suo utilizzo, diffusione, copia o archiviazione sono proibite. Quindi, se avete ricevuto questo messaggio per errore, per cortesia ci informi rispondendo immediatamente a questa email e cancelli il suo contenuto

2 5

Support for netflow logs
by Raghunath Adhyapak 04 Dec '19

04 Dec '19

Hi, I was trying to receive Netflow logs from firewall devices in syslog-ng and then forward to a central server. Does syslog-ng support netflow such that I can validate and filter out all non-netflow log lines? I also dumped some netflow logs to a file and found it to be binary. Therefore I haven't been able to ascertain the format and filtering mechanism. Any pointers on this topic would be helpful. Thanks Raghu

4 7

Its possible compress file using syslog-ng
by William Luiz Ribeiro Vasconcelos Da Silva 03 Dec '19

03 Dec '19

How to configure the syslog-ng.conf to do compress file? Its possible to use this feature inside of file structure: file("/<dir>/${HOST}_${YEAR}${MONTH}${DAY}${HOUR}${MIN}${SEC}_GMT${TZ}_${UNIQID}.log" template("${PROGRAM};${MESSAGE};\n") owner(user) group(user) perm (0755) Tks, Esta mensagem, incluindo seus anexos, pode conter informações privilegiadas e/ou de caráter confidencial, não podendo ser retransmitida sem autorização do remetente. Se você não é o destinatário ou pessoa autorizada para recebê-la, informamos que o seu uso, divulgação, cópia ou arquivamento são proibidos. Portanto, se você recebeu esta mensagem por engano, por favor nos informe respondendo imediatamente a este e-mail e delete o seu conteúdo. This message, including its attachments, may contain privileged or confidential information, and it must not be fowarded without the express authorization of the sender. If you are not the intended recipient, we hereby inform you that the use, disclosure, copy or filing are forbidden. So, if you received this message as a mistake, please inform us by answering this e-mail and deleting its contents Questo messaggio, inclusi gli allegati, potrebbe contenere informazioni privilegiate e/o riservate, e non deve essere ritrasmesse senza l'autorizzazione del mittente. Se non siete il destinatario o la persona autorizzata a riceverlo, informiamo che il suo utilizzo, diffusione, copia o archiviazione sono proibite. Quindi, se avete ricevuto questo messaggio per errore, per cortesia ci informi rispondendo immediatamente a questa email e cancelli il suo contenuto

1 0