February 2019 - syslog-ng - lists.balabit.hu

Problems parsing Cisco syslogs
by N. Max Pierson 08 Mar '19

08 Mar '19

Hi List, I have been trying to get something in place that can parse syslogs from various Cisco devices. The message format is almost the same with a few exceptions. Here is what I have tried and it works but now it has created another problem I do not know how to troubleshoot. So that I could see exactly what was being parsed, I disabled the default parsing using the below. source s_network { udp(ip(0.0.0.0) port(514) flags(no-parse)); }; rewrite r_cisco{ subst('^<\d+>(\d+:|:)\s+(\.\w+|\w+)\s+\d+\s+\d+\s\d+:\d+:\d+:\s|^<\d+>:\s+\d+\s+\w+\s+\d+\s+\d+:\d+:\d+\s\w+:\s|^<\d+>(\d+:|:)\s', "", value("MESSAGE"), type("pcre"), flags("ignore-case")); }; destination d_mysql { sql(type(mysql) host("127.0.0.1") username("syslog-ng") password("password") database("syslog") table("messages_${HOST}") columns("datetime datetime", "host varchar(50)", "level varchar(10)", "message text") values("${R_YEAR}-${R_MONTH}-${R_DAY} ${R_HOUR}:${R_MIN}:${R_SEC}", "${HOST}", "${LEVEL}", "${MESSAGE}") indexes("datetime", "level") ); }; log { source(s_network); rewrite(r_cisco); destination(d_mysql); }; This works perfectly as it formats the message as I want and covers IOS and NX-OS devices. The problem is when I turned off the default parser, now all of my logs show "notice" in the $LEVEL macro and doesn't reflect the real message header level. The $HOST macro still works fine however. Is this the expected behavior that the message header fields are not parsed as well as the $MESSAGE itself not being parsed? How can map the header level field properly to the $LEVEL marco if I disable the default parser? Regards, Max

7 20

How to use regex in syslog-ng.conf
by Lin, Victor 01 Mar '19

01 Mar '19

Dear all, I am trying to use regex in syslog-ng.conf without success :( Below is from my filter match("cmd=username toto privilege 15", value("MESSAGE")); could you please let me know how could I replace username toto with regex ? tried /w+ , but didn't passing through Thank you very much for your instruction! VL _______________________________________________________________________ If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference. Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.

3 2

OSE 3.20.1
by Szakacs, Attila 28 Feb '19

28 Feb '19

3.20.1Features - Add persist-tool (#2511 <https://github.com/balabit/syslog-ng/pull/2511>) - Collectd destination (#1433 <https://github.com/balabit/syslog-ng/pull/1433>) - Config reload status feedback (#2367 <https://github.com/balabit/syslog-ng/pull/2367>) - Netskope parser (#2541 <https://github.com/balabit/syslog-ng/pull/2541> ) - Websense parser (#2471 <https://github.com/balabit/syslog-ng/pull/2471> ) - Json list support (#2536 <https://github.com/balabit/syslog-ng/pull/2536>) - Xml-parser: add list support (#2544 <https://github.com/balabit/syslog-ng/pull/2544>) Bugfixes - DNS memory leak/segfault fix (#2500 <https://github.com/balabit/syslog-ng/pull/2500>) - Elasticsearch2: fix bulk send for client-mode("http") (#2478 <https://github.com/balabit/syslog-ng/pull/2478>) - Few leak fix (#2563 <https://github.com/balabit/syslog-ng/pull/2563>) - Filter-re: ref/unref NVTable around regex eval (#2494 <https://github.com/balabit/syslog-ng/pull/2494>) - Fix allowing negative version number in config (#2548 <https://github.com/balabit/syslog-ng/pull/2548>) - Fix app-parser() per reload memory leak (#2469 <https://github.com/balabit/syslog-ng/pull/2469>) - Fix non-reliable disk-queue truncating problem on load (#2406 <https://github.com/balabit/syslog-ng/pull/2406>) - Fix threaded source/destination crash when reverting configuration ( #2555 <https://github.com/balabit/syslog-ng/pull/2555>) - Http: add missing free for self->body_template (#2558 <https://github.com/balabit/syslog-ng/pull/2558>) - Java, elasticsearch2, explicit unsupport for named templates (#2534 <https://github.com/balabit/syslog-ng/pull/2534>) - Loggen: parameter handling (#2477 <https://github.com/balabit/syslog-ng/pull/2477>) - Python-source: fix crash when posting from python thread (#2533 <https://github.com/balabit/syslog-ng/pull/2533>) - Read acknowledgments send by Riemann (#2523 <https://github.com/balabit/syslog-ng/pull/2523>) - Redis, Riemann: fix ref/unref-ing templates (#2514 <https://github.com/balabit/syslog-ng/pull/2514>, #2530 <https://github.com/balabit/syslog-ng/pull/2530>) - Syslog-ng@default: use pid file location on control socket (#2489 <https://github.com/balabit/syslog-ng/pull/2489>) - Threaded-dest: Fix integer overflow (#2512 <https://github.com/balabit/syslog-ng/pull/2512>) - Threaded-dest: make persist_name local (#2516 <https://github.com/balabit/syslog-ng/pull/2516>) - Xml/geoip2: make prefix optional (#2538 <https://github.com/balabit/syslog-ng/pull/2538>) Other changes - Autotools, cmake: add detection for pylint, nosetests (#2537 <https://github.com/balabit/syslog-ng/pull/2537>,#2564 <https://github.com/balabit/syslog-ng/pull/2564>) - Autotools: relaxing python dependency requirement (#2472 <https://github.com/balabit/syslog-ng/pull/2472>) - Cfg-parser: printing error positions in case of parse failure (#2455 <https://github.com/balabit/syslog-ng/pull/2455>) - Cmake: add missing detections (#2510 <https://github.com/balabit/syslog-ng/pull/2510>) - Collect-cov.sh: make coverage should work with lcov in bionic (#2515 <https://github.com/balabit/syslog-ng/pull/2515>) - Configure: fix "--with-docbook*" option usage (#2465 <https://github.com/balabit/syslog-ng/pull/2465>) - Custom LGTM.com query for detecting uses of gmtime that are not covered by a lock (#2413 <https://github.com/balabit/syslog-ng/pull/2413> ) - Fix static uClibc-ng support (#2501 <https://github.com/balabit/syslog-ng/pull/2501>) - Format non-literal fixes (#2567 <https://github.com/balabit/syslog-ng/pull/2567>) - Grammar: remove the pragma less include (#2550 <https://github.com/balabit/syslog-ng/pull/2550>) - Http-worker: refactor & fix curl_easy_getinfo error message (#2527 <https://github.com/balabit/syslog-ng/pull/2527>) - Lib/compat: replace strcasestr() implementation (#2482 <https://github.com/balabit/syslog-ng/pull/2482>) - Libtest: Adding stopwatch.h into libtest_HEADERS list (#2553 <https://github.com/balabit/syslog-ng/pull/2553>) - Packaging: fix the description of mod-examples (#2522 <https://github.com/balabit/syslog-ng/pull/2522>) - Python-debugger: fix macro completion (#2439 <https://github.com/balabit/syslog-ng/pull/2439>) - Python, java destination add already introduced retry (#2559 <https://github.com/balabit/syslog-ng/pull/2559>) - Remove elastic v1 support (#2554 <https://github.com/balabit/syslog-ng/pull/2554>) - Simplify libtest queue utils (#2556 <https://github.com/balabit/syslog-ng/pull/2556>) - Syslog-ng.8.xml: remove unneeded default-modules section (#2475 <https://github.com/balabit/syslog-ng/pull/2475>) - Travis: use the latest Bison version (macOS) (#2529 <https://github.com/balabit/syslog-ng/pull/2529>) - Various fixes for issues reported by LGTM (#2524 <https://github.com/balabit/syslog-ng/pull/2524>) Notes to the developers - Example-msg-generator: num option (#2565 <https://github.com/balabit/syslog-ng/pull/2565>) - Own grammar support in generator plugin (#2552 <https://github.com/balabit/syslog-ng/pull/2552>) - ProtoClient: provide process_in function to logwriter (#2468 <https://github.com/balabit/syslog-ng/pull/2468>) - Pytest_framework: eliminating __registered_instances, exposing SyslogNgCtl to user api (#2503 <https://github.com/balabit/syslog-ng/pull/2503>) - Pytest_framework: MessageReader: Explain the local context around python asserts (#2507 <https://github.com/balabit/syslog-ng/pull/2507>) - Pytest_framework: support for implicit groups in config.create_logpath (#2490 <https://github.com/balabit/syslog-ng/pull/2490>) - Pytest: Renaming pytest_framework to python_functional (#2542 <https://github.com/balabit/syslog-ng/pull/2542>) - Python-destination: send can return worker_insert_result_t, flush support (#2487 <https://github.com/balabit/syslog-ng/pull/2487>) - Python: internal() source exposed via syslogng.Logger (#2505 <https://github.com/balabit/syslog-ng/pull/2505>) - Remove unused submodules (#2525 <https://github.com/balabit/syslog-ng/pull/2525>) - Simpler names for WORKER_INSERT_RESULT_T in language bindings (#2506 <https://github.com/balabit/syslog-ng/pull/2506>) - Split xml-parser into xml-parser and xml-scanner (#2459 <https://github.com/balabit/syslog-ng/pull/2459>) Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Andras Mitzki, Andrej Valek, Antal Nemes, Attila Szakacs, Balazs Scheidler, Bas van Schaik, Fᴀʙɪᴇɴ Wᴇʀɴʟɪ, Gabor Nagy, Laszlo Boszormenyi, Laszlo Budai, Lorand Muzamel, László Várady, Mehul Prajapati, Naveen Revanna, Peter Czanik, Peter Kokai, Romain Tartière, Stephen, Terez Nemes, Norbert Takács, Soubhik Chakraborty, NottyRu, Chris Packham. https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.20.1 Best regards, Attila Szakacs

1 0

Re: [syslog-ng] syslog-ng Digest, Vol 166, Issue 14
by Lin, Victor 27 Feb '19

27 Feb '19

Thanks a lot Kokan!!!!! After I put the individual name for each persist-name, no error any more :-). Have a great day! VL -----Original Message----- From: syslog-ng [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of syslog-ng-request(a)lists.balabit.hu Sent: 2019, February, 27 3:25 AM To: syslog-ng(a)lists.balabit.hu Subject: syslog-ng Digest, Vol 166, Issue 14 Send syslog-ng mailing list submissions to syslog-ng(a)lists.balabit.hu To subscribe or unsubscribe via the World Wide Web, visit https://lists.balabit.hu/mailman/listinfo/syslog-ng or, via email, send a message with subject or body 'help' to syslog-ng-request(a)lists.balabit.hu You can reach the person managing the list at syslog-ng-owner(a)lists.balabit.hu When replying, please edit your Subject line so it is more specific than "Re: Contents of syslog-ng digest..." Today's Topics: 1. Re: syslog-ng.conf error (Péter) ---------------------------------------------------------------------- Message: 1 Date: Wed, 27 Feb 2019 09:24:50 +0100 From: Péter, Kókai <peter.kokai(a)oneidentity.com> To: "Syslog-ng users' and developers' mailing list" <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] syslog-ng.conf error Message-ID: <CABxQCpj_YRnzzkskbCoi_CULFEuNskO=X1OpsKrQ0T=0V3crug(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hello, In order to point out the exact location of issue, you have to share your configuration. If you do not want to share it, there is two hint to find it yourself: 1) The message "Error checking the uniqueness of the persist names, please override it with persist-name option. Shutting" actually truncated (possible by journalctl), and the full version should contain the location of the item that has conflicting persist-name. 2) About persist-name, syslog-ng uses a persist file to store some information (for example to keep data between restart/reload). Date is stored in that file via its key, those keys are generated based on the configuration item options (for file source it could be the path of the file, etc...), sometimes it could collide with other configured item and syslog-ng cannot resolve the name collition; but the user could with specifying the *persist-name* option, which override the default generated option. An example: source s { file("/tmp/file" persist-name("my-tmp-file") ); }; -- Kokan On Tue, Feb 26, 2019 at 9:57 PM Lin, Victor <victor.lin(a)rbc.com> wrote: > Dear all, > > I am a newbie for syslog-ng J, could you please help me to figure > out my issues with syslog-ng.conf ? > > After modified syslog-ng.conf by adding more source, destination and > filter, try to restart syslog-ng by /etc/init.d/syslog-ng restart > > Got the following error > ********************************************* > # */etc/init.d/syslog-ng restart* > Restarting syslog-ng (via systemctl): Job for syslog-ng.service > failed because the control process exited with error code. See > "systemctl status syslog-ng.service" and "journalctl -xe" for details. > [FAILED] > > **************************************** > # *systemctl status syslog-ng.service* â— syslog-ng.service - LSB: > start and stop syslog-NG daemon > Loaded: loaded (/etc/rc.d/init.d/syslog-ng; bad; vendor preset: > enabled > ) > Active: failed (Result: exit-code) since Tue 2019-02-26 15:17:04 > EST; 30s ago > Docs: man:systemd-sysv-generator(8) > Process: 53303 ExecStop=/etc/rc.d/init.d/syslog-ng stop > (code=exited, > status=0/SUCCESS) > Process: 292500 ExecReload=/etc/rc.d/init.d/syslog-ng reload > (code=exited, status=0/SUCCESS) > Process: 53320 ExecStart=/etc/rc.d/init.d/syslog-ng start > (code=exited, > status=1/FAILURE) > Main PID: 116276 > > Feb 26 15:17:04 ABCD.com syslog-ng[53320]: > [2019-02-26T15:17:04.083451] > WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; > Feb 26 15:17:04 ABCD.com syslog-ng[53320]: > [2019-02-26T15:17:04.083503] > WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; > Feb 26 15:17:04 ABCD.com syslog-ng[53320]: > [2019-02-26T15:17:04.083554] > WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; > Feb 26 15:17:04 ABCD.com syslog-ng[53320]: > [2019-02-26T15:17:04.250946] Error checking the uniqueness of the > persist names, please override it with persist-name op...conf:131:9' > Feb 26 15:17:04 ABCD.com syslog-ng[53320]: > [2019-02-26T15:17:04.250984] Error checking the uniqueness of the > persist names, please override it with persist-name op...conf:132:9' > Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [FAILED] Feb 26 15:17:04 > ABCD.com systemd[1]: syslog-ng.service: control process exited, > code=exited status=1 Feb 26 15:17:04 ABCD.com systemd[1]: Failed to > start LSB: start and stop syslog-NG daemon. > Feb 26 15:17:04 ABCD.com systemd[1]: Unit syslog-ng.service entered > failed state. > Feb 26 15:17:04 ABCD.com systemd[1]: syslog-ng.service failed. > Hint: Some lines were ellipsized, use -l to show in full. > > > # *journalctl -xe* > Feb 26 15:05:28 ABCD.com puppet-agent[48945]: Could not send report: > getaddrinfo: No address associated with hostname Feb 26 15:09:40 > ABCD.com sshd[51578]: pam_sss(sshd:auth): authentication success; > logname= uid=0 euid=0 tty=ssh ruser= rhost= > m3vrb434.rbc1.royalbank.com user=VL51 Feb 26 15:09:40 ABCD.com > sshd[51578]: Accepted password for VL51 from > 10.94.201.168 port 62031 ssh2 > Feb 26 15:10:02 ABCD.com crond[51740]: pam_unix(crond:account): > password for user root will expire in 8 days Feb 26 15:10:02 ABCD.com > crond[51741]: pam_unix(crond:account): password for user root will > expire in 8 days Feb 26 15:10:02 ABCD.com crond[51740]: > pam_unix(crond:session): session opened for user root by (uid=0) Feb > 26 15:10:02 ABCD.com crond[51741]: pam_unix(crond:session): session > opened for user root by (uid=0) Feb 26 15:10:02 ABCD.com CROND[51743]: > (root) CMD (/usr/lib64/sa/sa1 1 1) Feb 26 15:10:02 ABCD.com > CROND[51742]: (root) CMD (/bin/printf > '\n******************************************************************* > ****************\n\n > For filesize:' >> /app/ > Feb 26 15:10:02 ABCD.com CROND[51740]: pam_unix(crond:session): > session closed for user root Feb 26 15:10:02 ABCD.com sendmail[51759]: > x1QKA2Bw051759: from=root, size=1598, class=-60, nrcpts=1, msgid=< > *201902262010.x1QKA2Bw051759(a)ABCD.com* > <201902262010.x1QKA2Bw051759(a)ABCD.com>>, relay=root@localho Feb 26 > 15:10:02 ABCD.com sendmail[51759]: x1QKA2Bw051759: to=root, > ctladdr=root (0/0), delay=00:00:00, xdelay=00:00:00, mailer=relay, > pri=139598, relay=[127.0.0.1] [127.0.0.1], Feb 26 15:10:02 ABCD.com > CROND[51741]: pam_unix(crond:session): session closed for user root > Feb 26 15:10:50 ABCD.com sudo[51905]: VL51 : TTY=pts/0 ; > PWD=/app/syslog-ng/custom/conf ; USER=root ; COMMAND=/usr/bin/su - Feb > 26 15:10:50 ABCD.com su[51909]: (to root) VL51 on pts/0 Feb 26 > 15:10:50 ABCD.com su[51909]: pam_unix(su-l:session): session opened > for user root by VL51(uid=0) Feb 26 15:15:01 ABCD.com crond[52764]: > pam_unix(crond:account): password for user root will expire in 8 days > Feb 26 15:15:01 ABCD.com crond[52764]: pam_unix(crond:session): > session opened for user root by (uid=0) Feb 26 15:15:01 ABCD.com > CROND[52765]: (root) CMD (/usr/sbin/logrotate > /app/syslog-ng/custom/conf/syslog-ng-rotate.conf) > Feb 26 15:15:01 ABCD.com CROND[52764]: pam_unix(crond:session): > session closed for user root Feb 26 15:17:02 ABCD.com polkitd[1117]: > Registered Authentication Agent for unix-process:53295:2185270730 > (system bus name :1.268 [/usr/bin/pkttyagent --notify-fd 5 > --fallback], Feb 26 15:17:02 ABCD.com systemd[1]: Stopping LSB: start > and stop syslog-NG daemon... > -- Subject: Unit syslog-ng.service has begun shutting down > -- Defined-By: systemd > -- Support: > *http://lists.freedesktop.org/mailman/listinfo/systemd-devel* > <http://lists.freedesktop.org/mailman/listinfo/systemd-devel> > -- > -- Unit syslog-ng.service has begun shutting down. > Feb 26 15:17:02 ABCD.com supervise/syslog-ng[116275]: Daemon exited > gracefully, not restarting; exitcode='0' > Feb 26 15:17:03 ABCD.com syslog-ng[53303]: Stopping syslog-ng: [ OK > ] Feb 26 15:17:03 ABCD.com systemd[1]: Starting LSB: start and stop > syslog-NG daemon... > -- Subject: Unit syslog-ng.service has begun start-up > -- Defined-By: systemd > -- Support: > *http://lists.freedesktop.org/mailman/listinfo/systemd-devel* > <http://lists.freedesktop.org/mailman/listinfo/systemd-devel> > -- > -- Unit syslog-ng.service has begun starting up. > Feb 26 15:17:04 ABCD.com syslog-ng[53320]: Starting syslog-ng: > [2019-02-26T15:17:04.082875] WARNING: With use-dns(no), dns-cache() > will be forced to 'no' too!; Feb 26 15:17:04 ABCD.com > syslog-ng[53320]: [2019-02-26T15:17:04.083333] > WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; > Feb 26 15:17:04 ABCD.com syslog-ng[53320]: > [2019-02-26T15:17:04.083396] > WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; > Feb 26 15:17:04 ABCD.com syslog-ng[53320]: > [2019-02-26T15:17:04.083451] > WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; > Feb 26 15:17:04 ABCD.com syslog-ng[53320]: > [2019-02-26T15:17:04.083503] > WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; > Feb 26 15:17:04 ABCD.com syslog-ng[53320]: > [2019-02-26T15:17:04.083554] > WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; > Feb 26 15:17:04 ABCD.com syslog-ng[53320]: > [2019-02-26T15:17:04.250946] Error checking the uniqueness of the > persist names, please override it with persist-name option. Shutting > Feb 26 15:17:04 ABCD.com syslog-ng[53320]: > [2019-02-26T15:17:04.250984] Error checking the uniqueness of the > persist names, please override it with persist-name option. Shutting > Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [FAILED] Feb 26 15:17:04 > ABCD.com systemd[1]: syslog-ng.service: control process exited, > code=exited status=1 Feb 26 15:17:04 ABCD.com systemd[1]: Failed to > start LSB: start and stop syslog-NG daemon. > -- Subject: Unit syslog-ng.service has failed > -- Defined-By: systemd > -- Support: > *http://lists.freedesktop.org/mailman/listinfo/systemd-devel* > <http://lists.freedesktop.org/mailman/listinfo/systemd-devel> > -- > -- Unit syslog-ng.service has failed. > -- > -- The result is failed. > Feb 26 15:17:04 ABCD.com systemd[1]: Unit syslog-ng.service entered > failed state. > Feb 26 15:17:04 ABCD.com systemd[1]: syslog-ng.service failed. > Feb 26 15:17:04 ABCD.com polkitd[1117]: Unregistered Authentication > Agent for unix-process:53295:2185270730 (system bus name :1.268, > object path /org/freedesktop/PolicyKit1/Auth Feb 26 15:20:01 ABCD.com > crond[53909]: pam_unix(crond:account): password for user root will > expire in 8 days Feb 26 15:20:01 ABCD.com crond[53908]: > pam_unix(crond:account): password for user root will expire in 8 days > Feb 26 15:20:01 ABCD.com crond[53909]: pam_unix(crond:session): > session opened for user root by (uid=0) Feb 26 15:20:01 ABCD.com > crond[53908]: pam_unix(crond:session): session opened for user root by > (uid=0) Feb 26 15:20:01 ABCD.com CROND[53910]: (root) CMD > (/usr/lib64/sa/sa1 1 1) Feb 26 15:20:01 ABCD.com CROND[53911]: (root) > CMD (/bin/printf > '\n******************************************************************* > ****************\n\n > For filesize:' >> /app/ > Feb 26 15:20:01 ABCD.com CROND[53908]: pam_unix(crond:session): > session closed for user root Feb 26 15:20:01 ABCD.com sendmail[53926]: > x1QKK1vA053926: from=root, size=1805, class=-60, nrcpts=1, msgid=< > *201902262020.x1QKK1vA053926(a)ABCD.com* > <201902262020.x1QKK1vA053926(a)ABCD.com>>, relay=root@localho Feb 26 > 15:20:01 ABCD.com sendmail[53926]: x1QKK1vA053926: to=root, > ctladdr=root (0/0), delay=00:00:00, xdelay=00:00:00, mailer=relay, > pri=139805, relay=[127.0.0.1] [127.0.0.1], Feb 26 15:20:01 ABCD.com > CROND[53909]: pam_unix(crond:session): session closed for user root > > > Thank you very much for your expertize and instruction!!!!! > > *VL* > > > > > > > ______________________________________________________________________ > _ > > If you received this email in error, please advise the sender (by > return email or otherwise) immediately. You have consented to receive > the attached electronically at the above-noted email address; please > retain a copy of this confirmation for future reference. > > Si vous recevez ce courriel par erreur, veuillez en aviser > l'expéditeur immédiatement, par retour de courriel ou par un autre > moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par > voie électronique à l'adresse courriel indiquée ci-dessus; veuillez > conserver une copie de cette confirmation pour les fins de reference future. > > > ______________________________________________________________________ > ________ Member info: > https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > > -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20190227/351df70d/a…> ------------------------------ Subject: Digest Footer _______________________________________________ syslog-ng maillist - syslog-ng(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng ------------------------------ End of syslog-ng Digest, Vol 166, Issue 14 ****************************************** _______________________________________________________________________ If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference. Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.

1 0

syslog-ng.conf error
by Lin, Victor 27 Feb '19

27 Feb '19

Dear all, I am a newbie for syslog-ng :), could you please help me to figure out my issues with syslog-ng.conf ? After modified syslog-ng.conf by adding more source, destination and filter, try to restart syslog-ng by /etc/init.d/syslog-ng restart Got the following error ********************************************* # /etc/init.d/syslog-ng restart Restarting syslog-ng (via systemctl): Job for syslog-ng.service failed because the control process exited with error code. See "systemctl status syslog-ng.service" and "journalctl -xe" for details. [FAILED] **************************************** # systemctl status syslog-ng.service â- syslog-ng.service - LSB: start and stop syslog-NG daemon Loaded: loaded (/etc/rc.d/init.d/syslog-ng; bad; vendor preset: enabled) Active: failed (Result: exit-code) since Tue 2019-02-26 15:17:04 EST; 30s ago Docs: man:systemd-sysv-generator(8) Process: 53303 ExecStop=/etc/rc.d/init.d/syslog-ng stop (code=exited, status=0/SUCCESS) Process: 292500 ExecReload=/etc/rc.d/init.d/syslog-ng reload (code=exited, status=0/SUCCESS) Process: 53320 ExecStart=/etc/rc.d/init.d/syslog-ng start (code=exited, status=1/FAILURE) Main PID: 116276 Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [2019-02-26T15:17:04.083451] WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [2019-02-26T15:17:04.083503] WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [2019-02-26T15:17:04.083554] WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [2019-02-26T15:17:04.250946] Error checking the uniqueness of the persist names, please override it with persist-name op...conf:131:9' Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [2019-02-26T15:17:04.250984] Error checking the uniqueness of the persist names, please override it with persist-name op...conf:132:9' Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [FAILED] Feb 26 15:17:04 ABCD.com systemd[1]: syslog-ng.service: control process exited, code=exited status=1 Feb 26 15:17:04 ABCD.com systemd[1]: Failed to start LSB: start and stop syslog-NG daemon. Feb 26 15:17:04 ABCD.com systemd[1]: Unit syslog-ng.service entered failed state. Feb 26 15:17:04 ABCD.com systemd[1]: syslog-ng.service failed. Hint: Some lines were ellipsized, use -l to show in full. # journalctl -xe Feb 26 15:05:28 ABCD.com puppet-agent[48945]: Could not send report: getaddrinfo: No address associated with hostname Feb 26 15:09:40 ABCD.com sshd[51578]: pam_sss(sshd:auth): authentication success; logname= uid=0 euid=0 tty=ssh ruser= rhost=m3vrb434.rbc1.royalbank.com user=VL51 Feb 26 15:09:40 ABCD.com sshd[51578]: Accepted password for VL51 from 10.94.201.168 port 62031 ssh2 Feb 26 15:10:02 ABCD.com crond[51740]: pam_unix(crond:account): password for user root will expire in 8 days Feb 26 15:10:02 ABCD.com crond[51741]: pam_unix(crond:account): password for user root will expire in 8 days Feb 26 15:10:02 ABCD.com crond[51740]: pam_unix(crond:session): session opened for user root by (uid=0) Feb 26 15:10:02 ABCD.com crond[51741]: pam_unix(crond:session): session opened for user root by (uid=0) Feb 26 15:10:02 ABCD.com CROND[51743]: (root) CMD (/usr/lib64/sa/sa1 1 1) Feb 26 15:10:02 ABCD.com CROND[51742]: (root) CMD (/bin/printf '\n***********************************************************************************\n\n For filesize:' >> /app/ Feb 26 15:10:02 ABCD.com CROND[51740]: pam_unix(crond:session): session closed for user root Feb 26 15:10:02 ABCD.com sendmail[51759]: x1QKA2Bw051759: from=root, size=1598, class=-60, nrcpts=1, msgid=<201902262010.x1QKA2Bw051759(a)ABCD.com<mailto:201902262010.x1QKA2Bw051759@ABCD.com>>, relay=root@localho Feb 26 15:10:02 ABCD.com sendmail[51759]: x1QKA2Bw051759: to=root, ctladdr=root (0/0), delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=139598, relay=[127.0.0.1] [127.0.0.1], Feb 26 15:10:02 ABCD.com CROND[51741]: pam_unix(crond:session): session closed for user root Feb 26 15:10:50 ABCD.com sudo[51905]: VL51 : TTY=pts/0 ; PWD=/app/syslog-ng/custom/conf ; USER=root ; COMMAND=/usr/bin/su - Feb 26 15:10:50 ABCD.com su[51909]: (to root) VL51 on pts/0 Feb 26 15:10:50 ABCD.com su[51909]: pam_unix(su-l:session): session opened for user root by VL51(uid=0) Feb 26 15:15:01 ABCD.com crond[52764]: pam_unix(crond:account): password for user root will expire in 8 days Feb 26 15:15:01 ABCD.com crond[52764]: pam_unix(crond:session): session opened for user root by (uid=0) Feb 26 15:15:01 ABCD.com CROND[52765]: (root) CMD (/usr/sbin/logrotate /app/syslog-ng/custom/conf/syslog-ng-rotate.conf) Feb 26 15:15:01 ABCD.com CROND[52764]: pam_unix(crond:session): session closed for user root Feb 26 15:17:02 ABCD.com polkitd[1117]: Registered Authentication Agent for unix-process:53295:2185270730 (system bus name :1.268 [/usr/bin/pkttyagent --notify-fd 5 --fallback], Feb 26 15:17:02 ABCD.com systemd[1]: Stopping LSB: start and stop syslog-NG daemon... -- Subject: Unit syslog-ng.service has begun shutting down -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has begun shutting down. Feb 26 15:17:02 ABCD.com supervise/syslog-ng[116275]: Daemon exited gracefully, not restarting; exitcode='0' Feb 26 15:17:03 ABCD.com syslog-ng[53303]: Stopping syslog-ng: [ OK ] Feb 26 15:17:03 ABCD.com systemd[1]: Starting LSB: start and stop syslog-NG daemon... -- Subject: Unit syslog-ng.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has begun starting up. Feb 26 15:17:04 ABCD.com syslog-ng[53320]: Starting syslog-ng: [2019-02-26T15:17:04.082875] WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [2019-02-26T15:17:04.083333] WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [2019-02-26T15:17:04.083396] WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [2019-02-26T15:17:04.083451] WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [2019-02-26T15:17:04.083503] WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [2019-02-26T15:17:04.083554] WARNING: With use-dns(no), dns-cache() will be forced to 'no' too!; Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [2019-02-26T15:17:04.250946] Error checking the uniqueness of the persist names, please override it with persist-name option. Shutting Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [2019-02-26T15:17:04.250984] Error checking the uniqueness of the persist names, please override it with persist-name option. Shutting Feb 26 15:17:04 ABCD.com syslog-ng[53320]: [FAILED] Feb 26 15:17:04 ABCD.com systemd[1]: syslog-ng.service: control process exited, code=exited status=1 Feb 26 15:17:04 ABCD.com systemd[1]: Failed to start LSB: start and stop syslog-NG daemon. -- Subject: Unit syslog-ng.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has failed. -- -- The result is failed. Feb 26 15:17:04 ABCD.com systemd[1]: Unit syslog-ng.service entered failed state. Feb 26 15:17:04 ABCD.com systemd[1]: syslog-ng.service failed. Feb 26 15:17:04 ABCD.com polkitd[1117]: Unregistered Authentication Agent for unix-process:53295:2185270730 (system bus name :1.268, object path /org/freedesktop/PolicyKit1/Auth Feb 26 15:20:01 ABCD.com crond[53909]: pam_unix(crond:account): password for user root will expire in 8 days Feb 26 15:20:01 ABCD.com crond[53908]: pam_unix(crond:account): password for user root will expire in 8 days Feb 26 15:20:01 ABCD.com crond[53909]: pam_unix(crond:session): session opened for user root by (uid=0) Feb 26 15:20:01 ABCD.com crond[53908]: pam_unix(crond:session): session opened for user root by (uid=0) Feb 26 15:20:01 ABCD.com CROND[53910]: (root) CMD (/usr/lib64/sa/sa1 1 1) Feb 26 15:20:01 ABCD.com CROND[53911]: (root) CMD (/bin/printf '\n***********************************************************************************\n\n For filesize:' >> /app/ Feb 26 15:20:01 ABCD.com CROND[53908]: pam_unix(crond:session): session closed for user root Feb 26 15:20:01 ABCD.com sendmail[53926]: x1QKK1vA053926: from=root, size=1805, class=-60, nrcpts=1, msgid=<201902262020.x1QKK1vA053926(a)ABCD.com<mailto:201902262020.x1QKK1vA053926@ABCD.com>>, relay=root@localho Feb 26 15:20:01 ABCD.com sendmail[53926]: x1QKK1vA053926: to=root, ctladdr=root (0/0), delay=00:00:00, xdelay=00:00:00, mailer=relay, pri=139805, relay=[127.0.0.1] [127.0.0.1], Feb 26 15:20:01 ABCD.com CROND[53909]: pam_unix(crond:session): session closed for user root Thank you very much for your expertize and instruction!!!!! VL _______________________________________________________________________ If you received this email in error, please advise the sender (by return email or otherwise) immediately. You have consented to receive the attached electronically at the above-noted email address; please retain a copy of this confirmation for future reference. Si vous recevez ce courriel par erreur, veuillez en aviser l'expéditeur immédiatement, par retour de courriel ou par un autre moyen. Vous avez accepté de recevoir le(s) document(s) ci-joint(s) par voie électronique à l'adresse courriel indiquée ci-dessus; veuillez conserver une copie de cette confirmation pour les fins de reference future.

2 1

Syslog-ng setup for both RFC3164 and RFC5124
by Carlan Philippe 27 Feb '19

27 Feb '19

Hi all, Is there a way to configure syslog-ng to process properly both RFC3164 and RFC5124 on the same listening port ? The scenario is a bunch of devices sending traffic to one syslog server port (both udp + tcp) with the senders typically not knowing what protocol they are sending. We are running syslog-ng 3.13 with this setup: source s_syslog { udp(ip(0.0.0.0) port(514)) ; tcp(ip(0.0.0.0) port(514)); } If needed we could upgrade syslog-ng to 3.19.1 but having checked the doc for 3.19, it seems that the solution would be to create 2 source entries, 1 for RFC3164 with network() and 1 for RFC5124 with syslog(). Neverthless, these 2 sources would have to listen on *different* ports and that is the problem for us. Note that we also have an identical issue with cisco traffic, since it's not RFC compliant, syslog-ng adds automatically a header with timestamp and hostname. Thank you.

4 3

Re: [syslog-ng] syslog-ng Digest, Vol 166, Issue 8
by Nathan Fish 25 Feb '19

25 Feb '19

Could you please clarify why the sources don't know what protocol they are sending? Are they relaying from other unknown sources? > Date: Sun, 24 Feb 2019 01:07:01 +0000 (UTC) > From: Carlan Philippe <philrmls(a)yahoo.fr> > To: "syslog-ng(a)lists.balabit.hu" <syslog-ng(a)lists.balabit.hu> > Subject: [syslog-ng] Syslog-ng setup for both RFC3164 and RFC5124 > Message-ID: <1313969407.6661190.1550970421020(a)mail.yahoo.com> > Content-Type: text/plain; charset="utf-8" > > Hi all, > Is there a way to configure syslog-ng to process properly both RFC3164 and RFC5124 on the same listening port ? > The scenario is a bunch of devices sending traffic to one syslog server port (both udp + tcp) with the senders typically not knowing what protocol they are sending. > We are running syslog-ng 3.13 with this setup: > source s_syslog { udp(ip(0.0.0.0) port(514)) ; tcp(ip(0.0.0.0) port(514)); } > > If needed we could upgrade syslog-ng to 3.19.1 but having checked the doc for 3.19, it seems that the solution would be to create 2 source entries, 1 for RFC3164 with network() and 1 for RFC5124 with syslog(). Neverthless, these 2 sources would have to listen on *different* ports and that is the problem for us. > Note that we also have an identical issue with cisco traffic, since it's not RFC compliant, syslog-ng adds automatically a header with timestamp and hostname. > > Thank you.

2 1

Relay log message through Application LoadBalancer
by Ronald Fenner 23 Feb '19

23 Feb '19

I'm trying to figure out how to relay a message through an AWS Application Load Balancer. I have the config setup to relay the message with: destination d_kafka_unstructured { network( "<dns-name>" port(9082) );}; Look at the tcpdump log I can see that the message is relayed but the ALB doesn't know how to handle it and responds with 400 sinceing the message isn't a HTTP Request. What would be the best way to wrap the message in a http like request that the ALB will understand. Is there an easy way to just tell syslog-ng to relay the message as HTTP or would I need to write something to add/strip the HTTP wrapper. Ronald Fenner Network Architect Game Circus LLC. rfenner(a)gamecircus.com

3 3

Latest syslog-ng RPM needed for RHEL7
by Harish Shetty 22 Feb '19

22 Feb '19

Hi I am planning to upgrade my syslog servers to RHEL7 , I am able to see syslog-ng-3.19.1.tar.gz. Can i get RPM for the same Regards Harish Shetty

2 1

Regex Irregularities
by N. Max Pierson 13 Feb '19

13 Feb '19

Hi List, I am having some weird issues with rewrite regex that I cannot explain. I am simply trying to filter out the first part of the message which has the date in this format. Feb 8 09:13:32 CST: (there is one space at the end) When I use the following syntax, it doesn't match as expected. ^\w+\s\d+\s\d+:\d+:\d+\s\w+:\s I know this is the correct pattern because it works just fine on www.regexpal.com. I did some further testing and I have narrowed it down to the below ... ^\w+ 8 09:55:54 CST: (this seemed to also remove the space behind the month) ^\w+\s 8 09:59:37 CST: (notice this is the exact same as the above without the beginning space) ^\w+\s\d+ Feb 8 10:07:04 CST: (doesn't match anything as though the space between Feb and 8 isn't there) ^\w+\d+ Feb 8 10:11:54 CST: (again doesn't match anything as though there is a space between Feb and 8) So it seems to be something either with \w word class or the + quantifier and it somehow eats the space behind it possibly?? I am running 3.19.1 on Centos 7. Can anyone test this to confirm it isn't just local to my install for whatever reason? Regards, Max

3 3