- syslog-ng - lists.balabit.hu

Re: [syslog-ng] How to get list of IPs
by james.r.hendrick 12 May '17

12 May '17

If you don't need all logs in a single file you could use $HOST or $HOST_FROM as part of the file name Sent from my Verizon, Samsung Galaxy smartphone -------- Original message --------From: Sathish Sundaravel <sathish.sundaravel(a)gmail.com> Date: 5/12/17 1:11 PM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: [syslog-ng] How to get list of IPs Hi All I have syslog Ng server running on Solaris platform and configured to receive or accept events from all unix servers in one single log file. If I want to collect list of servers reporting to the log server , I have to manually extract list from a single log file which is too much time consuming . What would the best way ? RegardsSathish

1 0

Syslog-ng input for beats ?
by Scot 12 May '17

12 May '17

Hi Has anyone tried using syslog-ng inputs for logstash or beats data ? I'm trying to find a solution that will let me mirror my beats data like syslog-ng lets me do with syslog traffic. my syslog data goes to file, es and splunk. Need the same split for winlogbeat Winlogbeat goes to logstash then ES. Thanks for reading.

4 21

Unable to track root activity in syslog-ng
by vijay amruth 11 May '17

11 May '17

Hello everyone, here is is my configuration file, I am unable to track root activity, I am able to track user activity like the commands ran etc. For example: If I run a command as sudo, I see it in the log however the same command when switched to root is not being tracked. Any help is appreciated. Thank you. @version:3.9 @include "scl.conf" options { threaded(yes); }; source s_sys { unix-stream("/dev/log"); system(); internal(); }; # Destinations ############## destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern" ); }; destination d_mlal { usertty("*"); }; # Filters ########## filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; # Log Bindings ############## #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; -- Thanks, Vijay.

4 4

Documentation error
by Evan Rempel 11 May '17

11 May '17

The documentation defines the macro WEEK_ABBREV, C_WEEK_ABBREV, R_WEEK_ABBREV, S_WEEK_ABBREV Description: The 3-letter English abbreviation of the name of the day the message was sent, for example Thu. but syslog-ng (3.9) actually populates the macros WEEK_DAY_ABBREV, C_WEEK_DAY_ABBREV, R_WEEK_DAY_ABBREV, S_WEEK_DAY_ABBREV -- Evan

2 1

unidirectional unix-stream
by Fabien Wernli 11 May '17

11 May '17

Hi, I'm trying to use the unix-stream destination to send data to another application which opens the socket in line mode. The problem is that both ends complain about the connection breaking all the time: syslog-ng[31002]: Syslog connection broken; fd='24', server='AF_UNIX(/var/run/otherapp.sock)', time_reopen='60' otherapp[123445]: failed to read from socket #10: Connection reset by peer I tracked down the problem to the fact that otherapp sends some data after receiving a line from syslog-ng: "data ok", and syslog-ng doesn't seem to like that. If I set up a unidirectional proxy (that filters out upstream "data ok") between syslog-ng and otherapp everything works fine: socat -u UNIX-LISTEN:/tmp/proxy.sock,fork UNIX-CONNECT:/var/run/otherapp.sock Am I doing something wrong here, or is this a known limitation of syslog-ng?

3 9

is it possible to use if() ... else statements on syslog-ng 3.7?
by Jorge Pereira 11 May '17

11 May '17

hi team, is it possible to use if() ... else statements on syslog-ng 3.7? I am trying to do something like. if ($value >= X) ...... else ...... -- Jorge Pereira

6 11

CDATA in patterndb test_message
by John Kristoff 08 May '17

08 May '17

I wonder if it might be possible to be able to instruct the patternsdb parser to ignore markup for the test_message. This is a minor quibble, but I do have some messages that I parse that could use something like a <![CDATA[]]> wrapper around the test_message text. If I do a 'pdbtool test --validate' against a database that includes the offending test_message I get an error, but if I wrap it in a CDATA it seems to run without error, although the parser doesn't actually seem to see that wrapped test message. I can also verify the test message with 'pdbtool match -M' command, so this isn't a big deal, just thought I'd bring attention to it and see if maybe I'm missing something or if this might be something to handle in a future release. John

1 0

syslog-ng wont start - please help
by vijay amruth 06 May '17

06 May '17

Hello Peter/ team, I am stuck with issue since four days, spent hours and hours, I am unable find a cause, I compiled syslog-ng from source and resolved all the dependencies, when I get to start it, it wont start. here is the out put, please have a look at it. Any help is appreciated. OS version : CentOS Linux release 7.3.1611 (Core) [root@my-host ~]# systemctl start syslog-ng.service Job for syslog-ng.service failed because a timeout was exceeded. See "systemctl status syslog-ng.service" and "journalctl -xe" for details. [root@my-host ~]# systemctl status syslog-ng.service ● syslog-ng.service - System Logger Daemon Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled) Active: activating (start) since Thu 2017-05-04 16:13:44 PDT; 29s ago Docs: man:syslog-ng(8) Main PID: 5089 (syslog-ng) CGroup: /system.slice/syslog-ng.service └─5089 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid May 04 16:13:44 my-host systemd[1]: Starting System Logger Daemon... [root@my-host ~]# journalctl -xe May 04 16:10:01 my-host systemd[1]: Starting Session 348 of user root. -- Subject: Unit session-348.scope has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit session-348.scope has begun starting up. May 04 16:10:21 my-host nslcd[1289]: [18832f] <group/member="root"> failed to bind to LDAP server ldap://ldap.vip.slc.e May 04 16:10:21 my-host nslcd[1289]: [18832f] <group/member="root"> no available LDAP server found: Can't contact LDAP May 04 16:10:21 my-host nslcd[1289]: [18832f] <group/member="root"> no available LDAP server found: Server is unavailab May 04 16:10:21 my-host CROND[5035]: (root) CMD (/usr/lib64/sa/sa1 1 1) May 04 16:10:23 my-host polkitd[676]: Registered Authentication Agent for unix-process:5039:17396299 (system bus name : May 04 16:10:23 my-host systemd[1]: Stopping Flush Journal to Persistent Storage... -- Subject: Unit systemd-journal-flush.service has begun shutting down -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit systemd-journal-flush.service has begun shutting down. May 04 16:10:23 my-host systemd-journal[2711]: Journal stopped -- Subject: The journal has been stopped -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- The system journal process has shut down and closed all currently -- active journal files. May 04 16:10:23 my-host systemd-journal[5046]: Runtime journal is using 16.0M (max allowed 391.1M, trying to leave 586. May 04 16:10:23 my-host systemd-journal[5046]: Runtime journal is using 16.0M (max allowed 391.1M, trying to leave 586. May 04 16:10:23 my-host systemd-journald[2711]: Received SIGTERM from PID 1 (systemd). May 04 16:10:23 my-host systemd-journal[5046]: Journal started -- Subject: The journal has been started -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- The system journal process has started up, opened the journal -- files for writing and is now ready to process requests. May 04 16:10:23 my-host polkitd[676]: Unregistered Authentication Agent for unix-process:5039:17396299 (system bus name May 04 16:10:23 my-host systemd[1]: Starting Flush Journal to Persistent Storage... -- Subject: Unit systemd-journal-flush.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit systemd-journal-flush.service has begun starting up. May 04 16:10:23 my-host systemd[1]: Started Flush Journal to Persistent Storage. -- Subject: Unit systemd-journal-flush.service has finished start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit systemd-journal-flush.service has finished starting up. -- -- The start-up result is done. May 04 16:12:14 my-host polkitd[676]: Registered Authentication Agent for unix-process:5066:17407401 (system bus name : May 04 16:12:14 my-host systemd[1]: Starting System Logger Daemon... -- Subject: Unit syslog-ng.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has begun starting up. May 04 16:13:08 my-host nslcd[1289]: [528108] <group/member="postfix"> failed to bind to LDAP server ldap://ldap.vip.sl May 04 16:13:08 my-host nslcd[1289]: [528108] <group/member="postfix"> no available LDAP server found: Can't contact LD May 04 16:13:08 my-host nslcd[1289]: [528108] <group/member="postfix"> no available LDAP server found: Server is unavai May 04 16:13:44 my-host systemd[1]: syslog-ng.service start operation timed out. Terminating. May 04 16:13:44 my-host systemd[1]: Failed to start System Logger Daemon. -- Subject: Unit syslog-ng.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has failed. -- -- The result is failed. May 04 16:13:44 my-host systemd[1]: Unit syslog-ng.service entered failed state. May 04 16:13:44 my-host systemd[1]: syslog-ng.service failed. May 04 16:13:44 my-host polkitd[676]: Unregistered Authentication Agent for unix-process:5066:17407401 (system bus name May 04 16:13:44 my-host systemd[1]: syslog-ng.service holdoff time over, scheduling restart. May 04 16:13:44 my-host systemd[1]: Starting System Logger Daemon... -- Subject: Unit syslog-ng.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has begun starting up. [root@my-host ~]# ps -ef | grep syslog-ng root 5089 1 0 16:13 ? 00:00:00 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid root 5103 32598 0 16:14 pts/1 00:00:00 grep --color=auto syslog-ng [root@my-host ~]# systemctl status syslog-ng.service ● syslog-ng.service - System Logger Daemon Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled) Active: activating (start) since Thu 2017-05-04 16:13:44 PDT; 1min 6s ago Docs: man:syslog-ng(8) Main PID: 5089 (syslog-ng) CGroup: /system.slice/syslog-ng.service └─5089 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid May 04 16:13:44 my-host systemd[1]: Starting System Logger Daemon... [root@my-host ~]#cat /usr/lib/systemd/system/syslog-ng.service [Unit] Description=System Logger Daemon Documentation=man:syslog-ng(8) [Service] Type=notify ExecStart=/usr/sbin/syslog-ng -F $SYSLOGNG_OPTS -p /var/run/syslogd.pid ExecReload=/bin/kill -HUP $MAINPID EnvironmentFile=-/etc/default/syslog-ng EnvironmentFile=-/etc/sysconfig/syslog-ng StandardOutput=journal StandardError=journal Restart=on-failure Environment=SYSTEMD_LOG_LEVEL=debug [Install] WantedBy=multi-user.target -- Thanks, Vijay Amrut.

2 5

PatternDB with CiscoSNMP Module
by Schoonover, Mark E HHHH 05 May '17

05 May '17

Thanks for reading, I'm in the process of creating a log to trap for Cisco devices. I have it working for facilities like this: TRINITY-2-TRINITY_SYSLOG_CRIT with this pattern: @ESTRING::%@@ESTRING:.cisco.Facility:-@@ESTRING:.cisco.Severity:-@@ESTRING:.cisco.MsgName::@ @ANYSTRING:.cisco.MsgText@ Now there are other facilities with an additional hyphen before the severity: EARL-DFC1-1-EXCESSIVE_PARITY_ERROR: Apr 26 14:04:45 hostname 565049: Apr 26 18:04:43.476 UTC: %EARL-DFC1-1-EXCESSIVE_PARITY_ERROR: EARL 0: Parity error detected in VRAM' which parses into: Testing message: program='EARL-DFC1-1-EXCESSIVE_PARITY_ERROR' message='Apr 26 14:04:45 hostname 565049: Apr 26 18:04:43.476 UTC: %EARL-DFC1-1-EXCESSIVE_PARITY_ERROR: EARL 0: Parity error detected in VRAM' Match name='.classifier.rule_id', value='09944c71-95eb-4bc0-8575-936931d85715', expected='09944c71-95eb-4bc0-8575-936931d85715' Wrong match name='.cisco.Facility', value='EARL', expected='EARL-DFC1' Wrong match name='.cisco.Severity', value='DFC1', expected='1' Wrong match name='.cisco.MsgName', value='1-EXCESSIVE_PARITY_ERROR', expected='EXCESSIVE_PARITY_ERROR' Match name='.cisco.MsgText', value='EARL 0: Parity error detected in VRAM', expected='EARL 0: Parity error detected in VRAM' I've tried a pattern that would parse EARL and DFC1 into separate variables, then append them into .cisco.Facility - which didn't work. Still that leaves the pattern hardcoded to just two hypens. I've also tried hard coding EARL-DFC1 into the pattern, then use value to set .cisco.Facility correctly. This approach works but being hardcoded, another Facility with embedded hyphens will fail. Ideally I'd like to have a pattern that matches on the last hyphen before the severity and capture that. I could parse using a regexp and setting $1, $2, $3, and $4 to the appropriate variables but I'm concerned about performance issues - I'm thinking this is a last resort solution. Any thoughts on how to proceed? Regards, Mark Schoonover - KA6WKE Infrastructure Engineering Manager, Splunk Architect ENE : Tools, Instrumentation and Common Services Team Office: 32.8697° N, 116.9711° W Phone : 770-261-7934 Email : mark.schoonover(a)cigna.com<mailto:mark.schoonover@cigna.com> HPSM Team: ENE NMS Engineering Confidential, unpublished property of CIGNA. Do not duplicate or distribute. Use and distribution limited solely to authorized personnel. ¬© Copyright 2017 CIGNA ------------------------------------------------------------------------------ CONFIDENTIALITY NOTICE: If you have received this email in error, please immediately notify the sender by e-mail at the address shown. This email transmission may contain confidential information. This information is intended only for the use of the individual(s) or entity to whom it is intended even if addressed incorrectly. Please delete it from your files if you are not the intended recipient. Thank you for your compliance. Copyright (c) 2017 Cigna ==============================================================================

4 5

unable to start syslog-ng.
by vijay amruth 01 May '17

01 May '17

Hello All, Hope you are all doing great. I am unable to start syslog-ng service Here is some information: *OS Version: Cent OS 7.3* *I compiled it from a tar ball..* [root@xxxxx system]# syslog-ng -V syslog-ng 3.9.1 Installer-Version: 3.9.1 Revision: Module-Directory: /usr/local/lib/syslog-ng Module-Path: /usr/local/lib/syslog-ng Available-Modules: kvformat,cef,disk-buffer,add-contextual-data,syslogformat,afsocket,affile,afprog,afuser,afamqp,afmongodb,csvparser,confgen,system-source,linux-kmsg-format,basicfuncs,cryptofuncs,dbparser,afstomp,pseudofile,graphite,sdjournal,date Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off *I manually added this file, this didn't come with install...* [root@xxxxx system]# cat syslog-ng.service [Unit] Description=System Logger Daemon Documentation=man:syslog-ng(8) [Service] Type=notify Sockets=syslog.socket ExecStart=/usr/sbin/syslog-ng -F -p /var/run/syslogd-ng.pid --fd-limit 50000 ExecReload=/bin/kill -HUP $MAINPID StandardOutput=null Restart=on-failure [Install] WantedBy=multi-user.target Alias=syslog.service [root@sl-sz3-splunk01 system]# pwd /lib/systemd/system *Had parser errors:* [root@xxxxxx ~]# syslog-ng -s Error parsing config, Error compiling template (Unknown template function "format-json") in /usr/local/share/syslog-ng/include/scl/cim/template.conf at line 23, column 32: included from /usr/local/etc/scl.conf line 29, column 1 included from /usr/local/etc/syslog-ng.conf line 8, column 1 template-function "format-cim" "$(format-json --pair @timestamp='${R_ISODATE}' --pair @message='${MSG}' --key .cim.* --shift 5 --key _* --key .* --replace-prefix .=_ --key *.*)\n"; ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng *Moved /usr/local/share/syslog-ng/include/scl/cim and no parser errors.* *But still unable to start.* [root@xxxxx system]# systemctl status syslog-ng.service ● syslog-ng.service - System Logger Daemon Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; disabled; vendor preset: enabled) Active: failed (Result: start-limit) since Mon 2017-05-01 10:46:58 PDT; 7s ago Docs: man:syslog-ng(8) Process: 2170 ExecStart=/usr/sbin/syslog-ng -F -p /var/run/syslogd-ng.pid --fd-limit 50000 (code=exited, status=203/EXEC) Main PID: 2170 (code=exited, status=203/EXEC) May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: syslog-ng.service: main process exited, code=exited, status=203/EXEC May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: Failed to start System Logger Daemon. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: Unit syslog-ng.service entered failed state. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: syslog-ng.service failed. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: syslog-ng.service holdoff time over, scheduling restart. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: start request repeated too quickly for syslog-ng.service May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: Failed to start System Logger Daemon. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: Unit syslog-ng.service entered failed state. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: syslog-ng.service failed. What I am I missing? Any help is appreciated. Thank you. -- Thanks, Vijay Amrut.

3 3