March 2019 - syslog-ng - lists.balabit.hu

Trigger dynamic action in syslog-ng
by Jim Hendrick 08 Mar '19

08 Mar '19

I was wondering if anyone has used syslog-ng to trigger some dynamic action based on logs. For example, if a certain threshold of messages happens in a time window, send an alert. LIke suppress () but more general actions. Or if a specific event happens, send *.debug from that system for 5 minutes. Or run a program to collect system data and send it along based on some condition. Not thinking SIEM functionality here, but maybe allow the log servers to be more dynamic around what actions they take for basic things. Thoughts? Thanks. Jim

3 3

Problems parsing Cisco syslogs
by N. Max Pierson 08 Mar '19

08 Mar '19

Hi List, I have been trying to get something in place that can parse syslogs from various Cisco devices. The message format is almost the same with a few exceptions. Here is what I have tried and it works but now it has created another problem I do not know how to troubleshoot. So that I could see exactly what was being parsed, I disabled the default parsing using the below. source s_network { udp(ip(0.0.0.0) port(514) flags(no-parse)); }; rewrite r_cisco{ subst('^<\d+>(\d+:|:)\s+(\.\w+|\w+)\s+\d+\s+\d+\s\d+:\d+:\d+:\s|^<\d+>:\s+\d+\s+\w+\s+\d+\s+\d+:\d+:\d+\s\w+:\s|^<\d+>(\d+:|:)\s', "", value("MESSAGE"), type("pcre"), flags("ignore-case")); }; destination d_mysql { sql(type(mysql) host("127.0.0.1") username("syslog-ng") password("password") database("syslog") table("messages_${HOST}") columns("datetime datetime", "host varchar(50)", "level varchar(10)", "message text") values("${R_YEAR}-${R_MONTH}-${R_DAY} ${R_HOUR}:${R_MIN}:${R_SEC}", "${HOST}", "${LEVEL}", "${MESSAGE}") indexes("datetime", "level") ); }; log { source(s_network); rewrite(r_cisco); destination(d_mysql); }; This works perfectly as it formats the message as I want and covers IOS and NX-OS devices. The problem is when I turned off the default parser, now all of my logs show "notice" in the $LEVEL macro and doesn't reflect the real message header level. The $HOST macro still works fine however. Is this the expected behavior that the message header fields are not parsed as well as the $MESSAGE itself not being parsed? How can map the header level field properly to the $LEVEL marco if I disable the default parser? Regards, Max

7 20

use loggen to insert content of a file into local log file
by Lin, Victor 07 Mar '19

07 Mar '19

Dear All, Could anyone can give an example of using loggen to insert content of a file into a local log file ? Thank you very much for you great help! VL _______________________________________________________________________ If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference. Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.

2 1

replacing log statements
by Sergey 06 Mar '19

06 Mar '19

Hello. "allow-config-dups" allow to replace some objects. As I understand log objects cann't have identifier and cann't be replaced therefore. Is it true or I wrong? It would be nice to be able to replace log object also. -- Regards, Sergey

4 5

Check if field in message and use default if not.
by Ronald Fenner 05 Mar '19

05 Mar '19

I generate a Kafka topic to send the message to by doing: topic("${topic}.${environment}") What I'd like to do if the field wasn't in the json message to provide a default value. Im looking for like a tertiary operator to use In the topic or perhaps a rewrite rule. I see the conditional expressions but from the docs it doesn't look like I can use it in the topic or a rewrite rule NOTE: My messages are json stings that get parsed and then imploded back to json as it's sent. Thanks Ronald Fenner Network Architect Game Circus LLC. rfenner(a)gamecircus.com

3 4

RFC 3195
by venkateswarlu vinjamuri 04 Mar '19

04 Mar '19

Hi All, I would like to know if there is any syslog-ng version that supports RFC-3195. If so, please let me know the version and any config related changes. Please also provide the links for the same. Regards Venkat

2 1

How to use regex in syslog-ng.conf
by Lin, Victor 04 Mar '19

04 Mar '19

Thanks a lot Kokan!!!!! I got the result :-) One more question For the following two %AAA-6-AAA_ACCOUNTING_MESSAGE: update:10.94.200.210@pts/0:syslogtest:deleted user victor %AAA-6-AAA_ACCOUNTING_MESSAGE: update:10.94.201.173@pts/0:syslogtest:added user victor I try to use the following regex to match the text in red color, it shows works. AAA-6-AAA_ACCOUNTING_MESSAGE: [a-zA-Z0-9]+:[0-9.]+@[a-zA-Z0-9]+\/[a-zA-Z0-9]+:[a-zA-Z0-9]+:[a-zA-Z]+ user Is there a simple way to math " update:10.94.200.210@pts/0:syslogtest:" Thank you very much again‼‼! VL -----Original Message----- From: syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of syslog-ng-request(a)lists.balabit.hu Sent: 2019, March, 01 7:00 AM To: syslog-ng(a)lists.balabit.hu Subject: syslog-ng Digest, Vol 167, Issue 1 Send syslog-ng mailing list submissions to syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu> To subscribe or unsubscribe via the World Wide Web, visit https://lists.balabit.hu/mailman/listinfo/syslog-ng or, via email, send a message with subject or body 'help' to syslog-ng-request(a)lists.balabit.hu<mailto:syslog-ng-request@lists.balabit.hu> You can reach the person managing the list at syslog-ng-owner(a)lists.balabit.hu<mailto:syslog-ng-owner@lists.balabit.hu> When replying, please edit your Subject line so it is more specific than "Re: Contents of syslog-ng digest..." Today's Topics: 1. unofficial syslog-ng 3.20 packages for Debian/Ubuntu (Laszlo Budai) 2. Re: How to use regex in syslog-ng.conf (Péter) 3. Re: How to use regex in syslog-ng.conf (Fabien Wernli) ---------------------------------------------------------------------- Message: 1 Date: Fri, 1 Mar 2019 10:09:03 +0000 From: Laszlo Budai <laszlo.budai(a)outlook.com<mailto:laszlo.budai@outlook.com>> To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>> Subject: [syslog-ng] unofficial syslog-ng 3.20 packages for Debian/Ubuntu Message-ID: <VI1PR0601MB2237CC24E8908466F6ABC1B38E760(a)VI1PR0601MB2237.eurprd06.prod.outlook.com<mailto:VI1PR0601MB2237CC24E8908466F6ABC1B38E760@VI1PR0601MB2237.eurprd06.prod.outlook.com>> Content-Type: text/plain; charset="iso-8859-1" Hi, syslog-ng 3.20.1[1] packages are available in OBS repo[2]. List of supported OSs: * Debian 8.0 * Debian 9.0 [including armv7l] * Ubuntu 14.04 * Ubuntu 16.04 * Ubuntu 16.10 * Ubuntu 17.04 * Ubuntu 17.10 * Ubuntu 18.04 * Ubuntu 18.10 Install ------- example: Debian 9.0 1. get release key wget -qO - http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… | sudo apt-key add - 2. add repo to APT sources eg.: /etc/apt/sources.list.d/syslog-ng-obs.list deb http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/Deb… ./ Then `apt-get update` and `apt-get install syslog-ng-core` Links -------- [1] https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.20.1 [2] https://build.opensuse.org/package/show/home:laszlo_budai:syslog-ng/syslog-… regards, Laszlo Budai -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20190301/2e2934b4/a…> ------------------------------ Message: 2 Date: Fri, 1 Mar 2019 11:34:00 +0100 From: Péter, Kókai <peter.kokai(a)oneidentity.com<mailto:peter.kokai@oneidentity.com>> To: "Syslog-ng users' and developers' mailing list" <syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>> Subject: Re: [syslog-ng] How to use regex in syslog-ng.conf Message-ID: <CABxQCpjDdn3JSwA1btkF7GZGLX_De0qGq+i9GtOcz8JWjhgpzA(a)mail.gmail.com<mailto:CABxQCpjDdn3JSwA1btkF7GZGLX_De0qGq+i9GtOcz8JWjhgpzA@mail.gmail.com>> Content-Type: text/plain; charset="utf-8" Hello, Based on your example one possible solution could be: match("cmd=username [a-z]+ privilege 15" value("MESSAGE")); You could also check out the syslog-ng administrator guide, it covers a lot of possibilities: https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edi… -- Kokan On Thu, Feb 28, 2019 at 3:50 PM Lin, Victor <victor.lin(a)rbc.com<mailto:victor.lin@rbc.com>> wrote: > Dear all, > > I am trying to use regex in syslog-ng.conf without success L > > Below is from my filter > > match("cmd=username toto privilege 15", value("MESSAGE")); > > could you please let me know how could I replace username toto with > regex ? tried /w+ , but didn’t passing through > > Thank you very much for your instruction! > > VL > > > > > ______________________________________________________________________ > _ > > If you received this email in error, please advise the sender (by > return email or otherwise) immediately. You have consented to receive > the attached electronically at the above-noted email address; please > retain a copy of this confirmation for future reference. > > Si vous recevez ce courriel par erreur, veuillez en aviser > l'expéditeur immédiatement, par retour de courriel ou par un autre > moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par > voie électronique à l'adresse courriel indiquée ci-dessus; veuillez > conserver une copie de cette confirmation pour les fins de reference future. > > > ______________________________________________________________________ > ________ Member info: > https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20190301/7921ceb3/a…> ------------------------------ Message: 3 Date: Fri, 1 Mar 2019 12:50:50 +0100 From: Fabien Wernli <wernli(a)in2p3.fr<mailto:wernli@in2p3.fr>> To: "Syslog-ng users' and developers' mailing list" <syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu>> Subject: Re: [syslog-ng] How to use regex in syslog-ng.conf Message-ID: <20190301115050.hs3d5vjf27a7lwfe(a)ccfawe.in2p3.fr<mailto:20190301115050.hs3d5vjf27a7lwfe@ccfawe.in2p3.fr>> Content-Type: text/plain; charset="iso-8859-1" On Fri, Mar 01, 2019 at 11:34:00AM +0100, Péter, Kókai wrote: > Hello, > > Based on your example one possible solution could be: > match("cmd=username [a-z]+ privilege 15" value("MESSAGE")); > > You could also check out the syslog-ng administrator guide, it covers > a lot of possibilities: > https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-sourc > e-edition/3.20/administration-guide/63#TOPIC-1122022 also, prefer single quotes over double quotes: will make escaping easier -------------- next part -------------- A non-text attachment was scrubbed... Name: smime.p7s Type: application/x-pkcs7-signature Size: 2801 bytes Desc: not available URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20190301/21433a6a/a…> ------------------------------ Subject: Digest Footer _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu<mailto:syslog-ng@lists.balabit.hu> https://lists.balabit.hu/mailman/listinfo/syslog-ng ------------------------------ End of syslog-ng Digest, Vol 167, Issue 1 ***************************************** _______________________________________________________________________ If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference. Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.

2 1

How to send test syslog message to a local log file
by Lin, Victor 04 Mar '19

04 Mar '19

Dear All, Could you please let me know can I do this? Would like to insert the following test syslog messages Feb 5 10:16:03 10.2.3.4 2019 Feb 5 15:16:03.560 UTC: %AUTHPRIV-5-SYSTEM_MSG: New user added with username victor - securityd Feb 5 10:16:03 10.2.3.4 2019 Feb 5 15:16:03.561 UTC: %AAA-6-AAA_ACCOUNTING_MESSAGE: update:10.24.3.46@pts/0:syslogtest:added user victor Feb 5 10:16:03 10.2.3.4 2019 Feb 5 15:16:03.561 UTC: %TACACS-6-TACACS_ACCOUNTING_MESSAGE: update: 10.24.3.46@pts/0: syslogtest: added user victor to a local log file /MyApp/syslog-ng/output/test.log Thank you so much for your help! VL _______________________________________________________________________ If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference. Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.

2 1

3.19.1 and "make check"
by Sergey 03 Mar '19

03 Mar '19

Hello. How can I use unit tests when building? As I see $ ./configure --help | grep "unit test" --enable-valgrind Whether to enable Valgrind on the unit tests I should enable "valgrind". Ok, I attempt to do it but it unsuccesful. valgrind is detected: checking whether to enable Valgrind on the unit tests... checking for valgrind... valgrind yes checking for Valgrind tool memcheck... memcheck checking for Valgrind tool helgrind... helgrind checking for Valgrind tool drd... drd checking for Valgrind tool exp-sgcheck... exp-sgcheck but isn't used: Test dependencies: Criterion : no Unit tests : no And expected: $ make check Makefile:21717: *** "Unit tests disabled". Stop. Or is availability of Criterion a necessary condition also ? btw, what is the "Criterion"? Is it https://github.com/Snaipe/Criterion ? -- Regards, Sergey

2 1

How to add a keyword into syslog
by Lin, Victor 02 Mar '19

02 Mar '19

Dear All, I would like to add the sources of syslog (for example CiscoIOS, CiscoNXOS, Arista) when I forward the received syslog to remote server, Could you please let me know how to do it? Below is from my current syslog-ng.conf destination d_Arista_logs { file("/MyApp/syslog-ng/output/Arista.log"); network("10.2.234.21" port(514) transport(udp) spoof_source(yes)); }; VL _______________________________________________________________________ If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference. Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.

1 0