May 2017 - syslog-ng - lists.balabit.hu

is it possible to use if() ... else statements on syslog-ng 3.7?
by Jorge Pereira 11 May '17

11 May '17

hi team, is it possible to use if() ... else statements on syslog-ng 3.7? I am trying to do something like. if ($value >= X) ...... else ...... -- Jorge Pereira

6 11

CDATA in patterndb test_message
by John Kristoff 08 May '17

08 May '17

I wonder if it might be possible to be able to instruct the patternsdb parser to ignore markup for the test_message. This is a minor quibble, but I do have some messages that I parse that could use something like a <![CDATA[]]> wrapper around the test_message text. If I do a 'pdbtool test --validate' against a database that includes the offending test_message I get an error, but if I wrap it in a CDATA it seems to run without error, although the parser doesn't actually seem to see that wrapped test message. I can also verify the test message with 'pdbtool match -M' command, so this isn't a big deal, just thought I'd bring attention to it and see if maybe I'm missing something or if this might be something to handle in a future release. John

1 0

syslog-ng wont start - please help
by vijay amruth 06 May '17

06 May '17

Hello Peter/ team, I am stuck with issue since four days, spent hours and hours, I am unable find a cause, I compiled syslog-ng from source and resolved all the dependencies, when I get to start it, it wont start. here is the out put, please have a look at it. Any help is appreciated. OS version : CentOS Linux release 7.3.1611 (Core) [root@my-host ~]# systemctl start syslog-ng.service Job for syslog-ng.service failed because a timeout was exceeded. See "systemctl status syslog-ng.service" and "journalctl -xe" for details. [root@my-host ~]# systemctl status syslog-ng.service ● syslog-ng.service - System Logger Daemon Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled) Active: activating (start) since Thu 2017-05-04 16:13:44 PDT; 29s ago Docs: man:syslog-ng(8) Main PID: 5089 (syslog-ng) CGroup: /system.slice/syslog-ng.service └─5089 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid May 04 16:13:44 my-host systemd[1]: Starting System Logger Daemon... [root@my-host ~]# journalctl -xe May 04 16:10:01 my-host systemd[1]: Starting Session 348 of user root. -- Subject: Unit session-348.scope has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit session-348.scope has begun starting up. May 04 16:10:21 my-host nslcd[1289]: [18832f] <group/member="root"> failed to bind to LDAP server ldap://ldap.vip.slc.e May 04 16:10:21 my-host nslcd[1289]: [18832f] <group/member="root"> no available LDAP server found: Can't contact LDAP May 04 16:10:21 my-host nslcd[1289]: [18832f] <group/member="root"> no available LDAP server found: Server is unavailab May 04 16:10:21 my-host CROND[5035]: (root) CMD (/usr/lib64/sa/sa1 1 1) May 04 16:10:23 my-host polkitd[676]: Registered Authentication Agent for unix-process:5039:17396299 (system bus name : May 04 16:10:23 my-host systemd[1]: Stopping Flush Journal to Persistent Storage... -- Subject: Unit systemd-journal-flush.service has begun shutting down -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit systemd-journal-flush.service has begun shutting down. May 04 16:10:23 my-host systemd-journal[2711]: Journal stopped -- Subject: The journal has been stopped -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- The system journal process has shut down and closed all currently -- active journal files. May 04 16:10:23 my-host systemd-journal[5046]: Runtime journal is using 16.0M (max allowed 391.1M, trying to leave 586. May 04 16:10:23 my-host systemd-journal[5046]: Runtime journal is using 16.0M (max allowed 391.1M, trying to leave 586. May 04 16:10:23 my-host systemd-journald[2711]: Received SIGTERM from PID 1 (systemd). May 04 16:10:23 my-host systemd-journal[5046]: Journal started -- Subject: The journal has been started -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- The system journal process has started up, opened the journal -- files for writing and is now ready to process requests. May 04 16:10:23 my-host polkitd[676]: Unregistered Authentication Agent for unix-process:5039:17396299 (system bus name May 04 16:10:23 my-host systemd[1]: Starting Flush Journal to Persistent Storage... -- Subject: Unit systemd-journal-flush.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit systemd-journal-flush.service has begun starting up. May 04 16:10:23 my-host systemd[1]: Started Flush Journal to Persistent Storage. -- Subject: Unit systemd-journal-flush.service has finished start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit systemd-journal-flush.service has finished starting up. -- -- The start-up result is done. May 04 16:12:14 my-host polkitd[676]: Registered Authentication Agent for unix-process:5066:17407401 (system bus name : May 04 16:12:14 my-host systemd[1]: Starting System Logger Daemon... -- Subject: Unit syslog-ng.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has begun starting up. May 04 16:13:08 my-host nslcd[1289]: [528108] <group/member="postfix"> failed to bind to LDAP server ldap://ldap.vip.sl May 04 16:13:08 my-host nslcd[1289]: [528108] <group/member="postfix"> no available LDAP server found: Can't contact LD May 04 16:13:08 my-host nslcd[1289]: [528108] <group/member="postfix"> no available LDAP server found: Server is unavai May 04 16:13:44 my-host systemd[1]: syslog-ng.service start operation timed out. Terminating. May 04 16:13:44 my-host systemd[1]: Failed to start System Logger Daemon. -- Subject: Unit syslog-ng.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has failed. -- -- The result is failed. May 04 16:13:44 my-host systemd[1]: Unit syslog-ng.service entered failed state. May 04 16:13:44 my-host systemd[1]: syslog-ng.service failed. May 04 16:13:44 my-host polkitd[676]: Unregistered Authentication Agent for unix-process:5066:17407401 (system bus name May 04 16:13:44 my-host systemd[1]: syslog-ng.service holdoff time over, scheduling restart. May 04 16:13:44 my-host systemd[1]: Starting System Logger Daemon... -- Subject: Unit syslog-ng.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has begun starting up. [root@my-host ~]# ps -ef | grep syslog-ng root 5089 1 0 16:13 ? 00:00:00 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid root 5103 32598 0 16:14 pts/1 00:00:00 grep --color=auto syslog-ng [root@my-host ~]# systemctl status syslog-ng.service ● syslog-ng.service - System Logger Daemon Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled) Active: activating (start) since Thu 2017-05-04 16:13:44 PDT; 1min 6s ago Docs: man:syslog-ng(8) Main PID: 5089 (syslog-ng) CGroup: /system.slice/syslog-ng.service └─5089 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid May 04 16:13:44 my-host systemd[1]: Starting System Logger Daemon... [root@my-host ~]#cat /usr/lib/systemd/system/syslog-ng.service [Unit] Description=System Logger Daemon Documentation=man:syslog-ng(8) [Service] Type=notify ExecStart=/usr/sbin/syslog-ng -F $SYSLOGNG_OPTS -p /var/run/syslogd.pid ExecReload=/bin/kill -HUP $MAINPID EnvironmentFile=-/etc/default/syslog-ng EnvironmentFile=-/etc/sysconfig/syslog-ng StandardOutput=journal StandardError=journal Restart=on-failure Environment=SYSTEMD_LOG_LEVEL=debug [Install] WantedBy=multi-user.target -- Thanks, Vijay Amrut.

2 5

PatternDB with CiscoSNMP Module
by Schoonover, Mark E HHHH 05 May '17

05 May '17

Thanks for reading, I'm in the process of creating a log to trap for Cisco devices. I have it working for facilities like this: TRINITY-2-TRINITY_SYSLOG_CRIT with this pattern: @ESTRING::%@@ESTRING:.cisco.Facility:-@@ESTRING:.cisco.Severity:-@@ESTRING:.cisco.MsgName::@ @ANYSTRING:.cisco.MsgText@ Now there are other facilities with an additional hyphen before the severity: EARL-DFC1-1-EXCESSIVE_PARITY_ERROR: Apr 26 14:04:45 hostname 565049: Apr 26 18:04:43.476 UTC: %EARL-DFC1-1-EXCESSIVE_PARITY_ERROR: EARL 0: Parity error detected in VRAM' which parses into: Testing message: program='EARL-DFC1-1-EXCESSIVE_PARITY_ERROR' message='Apr 26 14:04:45 hostname 565049: Apr 26 18:04:43.476 UTC: %EARL-DFC1-1-EXCESSIVE_PARITY_ERROR: EARL 0: Parity error detected in VRAM' Match name='.classifier.rule_id', value='09944c71-95eb-4bc0-8575-936931d85715', expected='09944c71-95eb-4bc0-8575-936931d85715' Wrong match name='.cisco.Facility', value='EARL', expected='EARL-DFC1' Wrong match name='.cisco.Severity', value='DFC1', expected='1' Wrong match name='.cisco.MsgName', value='1-EXCESSIVE_PARITY_ERROR', expected='EXCESSIVE_PARITY_ERROR' Match name='.cisco.MsgText', value='EARL 0: Parity error detected in VRAM', expected='EARL 0: Parity error detected in VRAM' I've tried a pattern that would parse EARL and DFC1 into separate variables, then append them into .cisco.Facility - which didn't work. Still that leaves the pattern hardcoded to just two hypens. I've also tried hard coding EARL-DFC1 into the pattern, then use value to set .cisco.Facility correctly. This approach works but being hardcoded, another Facility with embedded hyphens will fail. Ideally I'd like to have a pattern that matches on the last hyphen before the severity and capture that. I could parse using a regexp and setting $1, $2, $3, and $4 to the appropriate variables but I'm concerned about performance issues - I'm thinking this is a last resort solution. Any thoughts on how to proceed? Regards, Mark Schoonover - KA6WKE Infrastructure Engineering Manager, Splunk Architect ENE : Tools, Instrumentation and Common Services Team Office: 32.8697° N, 116.9711° W Phone : 770-261-7934 Email : mark.schoonover(a)cigna.com<mailto:mark.schoonover@cigna.com> HPSM Team: ENE NMS Engineering Confidential, unpublished property of CIGNA. Do not duplicate or distribute. Use and distribution limited solely to authorized personnel. ¬© Copyright 2017 CIGNA ------------------------------------------------------------------------------ CONFIDENTIALITY NOTICE: If you have received this email in error, please immediately notify the sender by e-mail at the address shown. This email transmission may contain confidential information. This information is intended only for the use of the individual(s) or entity to whom it is intended even if addressed incorrectly. Please delete it from your files if you are not the intended recipient. Thank you for your compliance. Copyright (c) 2017 Cigna ==============================================================================

4 5

unable to start syslog-ng.
by vijay amruth 01 May '17

01 May '17

Hello All, Hope you are all doing great. I am unable to start syslog-ng service Here is some information: *OS Version: Cent OS 7.3* *I compiled it from a tar ball..* [root@xxxxx system]# syslog-ng -V syslog-ng 3.9.1 Installer-Version: 3.9.1 Revision: Module-Directory: /usr/local/lib/syslog-ng Module-Path: /usr/local/lib/syslog-ng Available-Modules: kvformat,cef,disk-buffer,add-contextual-data,syslogformat,afsocket,affile,afprog,afuser,afamqp,afmongodb,csvparser,confgen,system-source,linux-kmsg-format,basicfuncs,cryptofuncs,dbparser,afstomp,pseudofile,graphite,sdjournal,date Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off *I manually added this file, this didn't come with install...* [root@xxxxx system]# cat syslog-ng.service [Unit] Description=System Logger Daemon Documentation=man:syslog-ng(8) [Service] Type=notify Sockets=syslog.socket ExecStart=/usr/sbin/syslog-ng -F -p /var/run/syslogd-ng.pid --fd-limit 50000 ExecReload=/bin/kill -HUP $MAINPID StandardOutput=null Restart=on-failure [Install] WantedBy=multi-user.target Alias=syslog.service [root@sl-sz3-splunk01 system]# pwd /lib/systemd/system *Had parser errors:* [root@xxxxxx ~]# syslog-ng -s Error parsing config, Error compiling template (Unknown template function "format-json") in /usr/local/share/syslog-ng/include/scl/cim/template.conf at line 23, column 32: included from /usr/local/etc/scl.conf line 29, column 1 included from /usr/local/etc/syslog-ng.conf line 8, column 1 template-function "format-cim" "$(format-json --pair @timestamp='${R_ISODATE}' --pair @message='${MSG}' --key .cim.* --shift 5 --key _* --key .* --replace-prefix .=_ --key *.*)\n"; ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng *Moved /usr/local/share/syslog-ng/include/scl/cim and no parser errors.* *But still unable to start.* [root@xxxxx system]# systemctl status syslog-ng.service ● syslog-ng.service - System Logger Daemon Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; disabled; vendor preset: enabled) Active: failed (Result: start-limit) since Mon 2017-05-01 10:46:58 PDT; 7s ago Docs: man:syslog-ng(8) Process: 2170 ExecStart=/usr/sbin/syslog-ng -F -p /var/run/syslogd-ng.pid --fd-limit 50000 (code=exited, status=203/EXEC) Main PID: 2170 (code=exited, status=203/EXEC) May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: syslog-ng.service: main process exited, code=exited, status=203/EXEC May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: Failed to start System Logger Daemon. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: Unit syslog-ng.service entered failed state. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: syslog-ng.service failed. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: syslog-ng.service holdoff time over, scheduling restart. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: start request repeated too quickly for syslog-ng.service May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: Failed to start System Logger Daemon. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: Unit syslog-ng.service entered failed state. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: syslog-ng.service failed. What I am I missing? Any help is appreciated. Thank you. -- Thanks, Vijay Amrut.

3 3

syslog-ng 3.9.1 erroneously matching regex
by Soumyadip Das Mahapatra 01 May '17

01 May '17

Hi folks, I raised https://github.com/balabit/syslog-ng/issues/1455 Could you guys help me troubleshooting the issue. It looks to be a buggy regex match. Thanks, Soumyadip

2 1