- syslog-ng - lists.balabit.hu

How to export syslogng to elasticsearch
by Mik J 19 May '17

19 May '17

Hello, I've read many documentation about how to export syslogng to elasticsearch but still don't get it # syslog-ng -V syslog-ng 3.7.2 Installer-Version: 3.7.2 Revision: Compile-Date: May 8 2017 10:54:55 Available-Modules: afprog,afsocket,afsql,afuser,basicfuncs,confgen,csvparser,dbparser,graphite,json-plugin,kvformat,linux-kmsg-format,pseudofile,syslogformat,system-source,affile,cryptofuncs Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off In syslog-ng.conf I need to add: @module mod-java @include "scl.conf" In scl.conf I have (I kept the default configuration) @define scl-root "`syslog-ng-data`/include/scl" @define include-path "`include-path`:`syslog-ng-data`/include" @include 'scl/*/*.conf' I don't understand really what should I configure. I also downloaded gradle # ls /usr/local/gradle-3.5 And Java # /usr/local/jdk-1.8.0/bin/java -version openjdk version "1.8.0_121" OpenJDK Runtime Environment (build 1.8.0_121-b13) OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode) Back to syslog-ng.conf I added # Test Elasticsearch filter f_MyTest { host("1.1.1.1"); }; log { source(s_net); filter(f_MyTest); destination(d_MyTest); }; destination d_elastic { elasticsearch( index("syslog-ng_${YEAR}.${MONTH}.${DAY}") type("test") cluster("syslog-ng") client_mode("transport") custom_id("${UNIQID}") flush-limit("10000") ); }; log { source(s_net); filter(f_MyTest); destination(d_elastic); flags(flow-control); }; But starting the daemon gives me this error # syslog-ng [2017-05-18T16:43:42.465496] Plugin module not found in 'module-path'; module-path='/usr/local/lib/syslog-ng', module='mod-java' Error parsing destination, destination plugin elasticsearch not found in /etc/syslog-ng/syslog-ng.conf at line 171, column 2: elasticsearch( ^^^^^^^^^^^^^

3 6

syslog-ng with SELinux in enforcing mode
by Czanik, Péter 19 May '17

19 May '17

Hi, Recently a large part of syslog-ng problem reports were related to SELinux not allowing access to some custom network ports or directories. You can read the basics of customizing SELinux (and Firewalld) rules for your environment in my latest blog: https://www.balabit.com/blog/using-syslog-ng-with-selinux-in-enforcing-mode/ Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

1 0

use of Syslog-ng to filter logs for forwarding to Splunk
by Stuart Martin 18 May '17

18 May '17

Hi We are looking to collect logs from servers and devices in our DMZ and then filter the logs of unnecessary information to then be sent to our internal Splunk instance. Would Syslog-ng OSE edition be capable of this task? I've started to read through your excellent documentation for the OSE edition, I was wondering what route I should take to configure it for the scenario described above? The link I am thinking is correct is the "To configure syslog-ng OSE as a relay that receives log messages from client hosts and forwards them to a central logserver, see Procedure 4.2, Configuring syslog-ng on server hosts<https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-…>." Does that sound correct? Many Thanks Kind Regards Stuart Martin Infrastructure Security Engineer Surrey Satellite Technology Ltd Tycho House, 20 Stephenson Road, Surrey Research Park, Guildford, GU2 7YE Tel: +44 (0)1483 803803 | Fax: +44 (0)1483 803804 | Email: s.martin(a)sstl.co.uk<mailto:s.martin@sstl.co.uk> www.sstl.co.uk<http://www.sstl.co.uk/> http://twitter.com/SurreySat P Please consider the environment before printing this e-mail This e-mail and any attachments may contain confidential and privileged information. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this e-mail and destroy any copies. Any dissemination or use of this information by a person other than the intended recipient is unauthorized and may be illegal. The information contained in this message is intended for the addressee only and may contain sensitive information. If you are not the addressee, please delete this message and notify the sender; you should not copy or distribute this message or disclose its contents to anyone. Any views or opinions expressed in this message are those of the individual(s) and not necessarily of the organisation. No reliance may be placed on this message without written confirmation from an authorised representative of its contents. No guarantee is implied that this message or any attachment is virus free or has not been intercepted and amended.

3 5

DMZ Relay Config (multiple ports)
by wiskbroom＠hotmail.com 18 May '17

18 May '17

Hello! I am trying to build a syslog-ng.conf for a DMZ relay that will listen on several TCP and UDP ports, all working just great. My config essentially captures traffic, then redirects each connection internally from itself. What happens if my internal host goes down? I am hoping to store locally if connectivity is down, but not sure where or how that would get defined. I need to keep a direct relationship between ports, as my internal syslog-ng treats these ports differently, so redirecting to the same as received is key. Below is my DMZ relay syslog-NG config file, any offers to make it better greatly appreciated. -Vadim -------------------------syslog-ng.conf------------------------------ @version:3.5 @include "scl.conf" @include "/etc/syslog-ng/conf.d/*.conf" options { time-reap(30); mark-freq(10); keep-hostname(yes); chain_hostnames (off); flush_lines (0); time_reopen (10); log_fifo_size (1000); use_dns (yes); use_fqdn (no); create_dirs (no); }; source s_udp-relay { udp(ip(0.0.0.0) port(514) so_rcvbuf(425984)); }; source s_tcp-relay-514 { tcp(ip(0.0.0.0) port(514) max-connections(250) so_rcvbuf(425984) log_iw_size(25000) so_keepalive(yes) log_fetch_limit(100)); }; source s_tcp-relay-7514 { tcp(ip(0.0.0.0) port(7514) max-connections(250) so_rcvbuf(425984) log_iw_size(25000) so_keepalive(yes) log_fetch_limit(100)); }; destination d_syslog_udp { udp("10.5.5.5" port(514)); }; destination d_syslog_tcp-514 { tcp("10.5.5.5" port(514)); }; destination d_syslog_tcp-7514 { tcp("10.5.5.5" port(7514)); }; log { source(s_udp-relay); destination(d_syslog_udp); }; log { source(s_tcp-relay-514); destination(d_syslog_tcp-514); }; log { source(s_tcp-relay-7514); destination(d_syslog_tcp-7514); };

3 2

syslog-ng-3.9.1-1 is crashing
by Harish Shetty 17 May '17

17 May '17

Hi Peter I have deployed syslog-ng-3.9.1-1.el6.x86_64 , where I am seeing one issue. Where there is disk usage in greater than 90%, It coredumps. Instead of that shouldnt it be handle the syslog-ng service gracefully? I have pasted part of core here and attached the coredump full file gdb) bt full #0 0x0000003b52a325e5 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x0000003b52a33dc5 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x0000003206c7506b in iv_fatal (fmt=<value optimized out>) at iv_fatal.c:57 ap = {{gp_offset = 8, fp_offset = 48, overflow_arg_area = 0x7f35c5aadac0, reg_save_area = 0x7f35c5aada00}} msg = "iv_init: can't initialize event dispatcher\000\006\062\000\ 000\000\177\000\000\bZ\003\000\000\000\000`KL\234\065\177\ 000\000?M006\062\000\000\000@֪\177\000\000@֪\177\000\000\ 177\000\000\b\000\000\000\000\000\000\000L\234\065\177\000\ 000L\234\065\177\000\000\000KL\234\065\177\000\000Ye006\062\000\000\000\001\ 000\000\000\000\000\000\000ت\177\000\000ت\177\000\000\177\000\000ت\177", '\000' <repeats 46 times>, "\004", '\000' <repeats 35 times>... #3 0x0000003206c76f2a in iv_fd_init (st=0x7f359c945850) at iv_fd.c:129 No locals. #4 0x0000003206c77499 in iv_init () at iv_main_posix.c:74 st = 0x7f359c945850 #5 0x0000003206c75b8e in iv_work_thread (_thr=0x996c540) at iv_work.c:177 thr = 0x996c540 pool = 0x1df6820 #6 0x0000003206c77def in iv_thread_handler (_thr=0x52933d0) at iv_thread_posix.c:142 __clframe = {__cancel_routine = 0x3206c77e50 <iv_thread_cleanup_handler>, __cancel_arg = 0x52933d0, __do_it = 1, __cancel_type = 0} thr = 0x52933d0 #7 0x0000003b53207aa1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. ---Type <return> to continue, or q <return> to quit--- #8 0x0000003b52ae8aad in clone () from /lib64/libc.so.6 No symbol table info available. Can you please check and let me know how can we fix this issue? Regards Harish shetty

4 9

Re: [syslog-ng] Syslog-ng input for beats ?
by Scot 17 May '17

17 May '17

I tried reading the mapping documentation in the Balabit Latest docs but I'm not connecting these dots, sorry. Can I use the templates for creating the ES index as a reference for json mapping in syslog-ng ? For example the attached winlogbeat.template.json which created the ES index.

2 3

Running into Error: Could not find Java SE Runtime Environment
by vijay amruth 17 May '17

17 May '17

Hello Peter/team/everyone, Post configure syslog-ng 3.9.1 on redhat 7.3, I am running to this error when running make, please look into this, any help is appreciated. GEN modules/java/syslog-ng-core/classes/org/syslog_ng/LogMessage.class Error: could not find libjava.so Error: Could not find Java SE Runtime Environment. make: *** [modules/java/syslog-ng-core/classes/org/syslog_ng/LogMessage.class] Error 2 Tried fixing path, didn't help...... [root@sml-collector02 syslog-ng-3.9.1]# export JRE_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.102-4.b14.el7.x86_64/jre [root@sml-collector02 syslog-ng-3.9.1]# export PATH=$PATH:$JRE_HOME/bin [root@sml-collector02 syslog-ng-3.9.1]# export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.102-4.b14.el7.x86_64 [root@sml-collector02 syslog-ng-3.9.1]# export JAVA_PATH=$JAVA_HOME [root@sml-collector02 syslog-ng-3.9.1]# export PATH=$PATH:$JAVA_HOME/bin Please look into it. Thank you. -- Thanks, Vijay Amrut.

1 0

in-list filter for substrings
by Clayton Dukes 16 May '17

16 May '17

Question: Is there are way to read in a file to match for substrings? Example: filter f_malware { in-list("/etc/syslog-ng/conf.d/malware.list", value("MSG")); }; rewrite rw_malware { set("Malware", value("PROGRAM") condition(filter(f_malware))); }; Then, given a string like: Dec 15 17:57:02 230.16.190.121 %ASA-5-304001: 67.16.126.205 Accessed URL 114.4.201.244:http://wvqwdhimen.jaqwejggvwxyangoqepw.tk/storageframe.html It would look at malware.list and see a line containing: wvqwdhimen.jaqwejggvwxyangoqepw.tk and set the Program name as "Malware" I see in the docs, it says ' Only exact matches are supported, partial and substring matches are not' so I don't think this will work. Is there a way to do anything like this? - Clayton

1 0

How to get list of IPs
by Sathish Sundaravel 13 May '17

13 May '17

Hi All I have syslog Ng server running on Solaris platform and configured to receive or accept events from all unix servers in one single log file. If I want to collect list of servers reporting to the log server , I have to manually extract list from a single log file which is too much time consuming . What would the best way ? Regards Sathish

5 6

PLEASE REMOVE MY EMAIL FROM THIS LIST...
by Alex Norton 13 May '17

13 May '17

THANKS ALEX

2 1

Re: [syslog-ng] How to get list of IPs
by james.r.hendrick 12 May '17

12 May '17

If you don't need all logs in a single file you could use $HOST or $HOST_FROM as part of the file name Sent from my Verizon, Samsung Galaxy smartphone -------- Original message --------From: Sathish Sundaravel <sathish.sundaravel(a)gmail.com> Date: 5/12/17 1:11 PM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: [syslog-ng] How to get list of IPs Hi All I have syslog Ng server running on Solaris platform and configured to receive or accept events from all unix servers in one single log file. If I want to collect list of servers reporting to the log server , I have to manually extract list from a single log file which is too much time consuming . What would the best way ? RegardsSathish

1 0

Syslog-ng input for beats ?
by Scot 12 May '17

12 May '17

Hi Has anyone tried using syslog-ng inputs for logstash or beats data ? I'm trying to find a solution that will let me mirror my beats data like syslog-ng lets me do with syslog traffic. my syslog data goes to file, es and splunk. Need the same split for winlogbeat Winlogbeat goes to logstash then ES. Thanks for reading.

4 21

Unable to track root activity in syslog-ng
by vijay amruth 11 May '17

11 May '17

Hello everyone, here is is my configuration file, I am unable to track root activity, I am able to track user activity like the commands ran etc. For example: If I run a command as sudo, I see it in the log however the same command when switched to root is not being tracked. Any help is appreciated. Thank you. @version:3.9 @include "scl.conf" options { threaded(yes); }; source s_sys { unix-stream("/dev/log"); system(); internal(); }; # Destinations ############## destination d_cons { file("/dev/console"); }; destination d_mesg { file("/var/log/messages"); }; destination d_auth { file("/var/log/secure"); }; destination d_mail { file("/var/log/maillog" flush_lines(10)); }; destination d_spol { file("/var/log/spooler"); }; destination d_boot { file("/var/log/boot.log"); }; destination d_cron { file("/var/log/cron"); }; destination d_kern { file("/var/log/kern" ); }; destination d_mlal { usertty("*"); }; # Filters ########## filter f_kernel { facility(kern); }; filter f_default { level(info..emerg) and not (facility(mail) or facility(authpriv) or facility(cron)); }; filter f_auth { facility(authpriv); }; filter f_mail { facility(mail); }; filter f_emergency { level(emerg); }; filter f_news { facility(uucp) or (facility(news) and level(crit..emerg)); }; filter f_boot { facility(local7); }; filter f_cron { facility(cron); }; # Log Bindings ############## #log { source(s_sys); filter(f_kernel); destination(d_cons); }; log { source(s_sys); filter(f_kernel); destination(d_kern); }; log { source(s_sys); filter(f_default); destination(d_mesg); }; log { source(s_sys); filter(f_auth); destination(d_auth); }; log { source(s_sys); filter(f_mail); destination(d_mail); }; log { source(s_sys); filter(f_emergency); destination(d_mlal); }; log { source(s_sys); filter(f_news); destination(d_spol); }; log { source(s_sys); filter(f_boot); destination(d_boot); }; log { source(s_sys); filter(f_cron); destination(d_cron); }; -- Thanks, Vijay.

4 4

Documentation error
by Evan Rempel 11 May '17

11 May '17

The documentation defines the macro WEEK_ABBREV, C_WEEK_ABBREV, R_WEEK_ABBREV, S_WEEK_ABBREV Description: The 3-letter English abbreviation of the name of the day the message was sent, for example Thu. but syslog-ng (3.9) actually populates the macros WEEK_DAY_ABBREV, C_WEEK_DAY_ABBREV, R_WEEK_DAY_ABBREV, S_WEEK_DAY_ABBREV -- Evan

2 1

unidirectional unix-stream
by Fabien Wernli 11 May '17

11 May '17

Hi, I'm trying to use the unix-stream destination to send data to another application which opens the socket in line mode. The problem is that both ends complain about the connection breaking all the time: syslog-ng[31002]: Syslog connection broken; fd='24', server='AF_UNIX(/var/run/otherapp.sock)', time_reopen='60' otherapp[123445]: failed to read from socket #10: Connection reset by peer I tracked down the problem to the fact that otherapp sends some data after receiving a line from syslog-ng: "data ok", and syslog-ng doesn't seem to like that. If I set up a unidirectional proxy (that filters out upstream "data ok") between syslog-ng and otherapp everything works fine: socat -u UNIX-LISTEN:/tmp/proxy.sock,fork UNIX-CONNECT:/var/run/otherapp.sock Am I doing something wrong here, or is this a known limitation of syslog-ng?

3 9

is it possible to use if() ... else statements on syslog-ng 3.7?
by Jorge Pereira 11 May '17

11 May '17

hi team, is it possible to use if() ... else statements on syslog-ng 3.7? I am trying to do something like. if ($value >= X) ...... else ...... -- Jorge Pereira

6 11

CDATA in patterndb test_message
by John Kristoff 08 May '17

08 May '17

I wonder if it might be possible to be able to instruct the patternsdb parser to ignore markup for the test_message. This is a minor quibble, but I do have some messages that I parse that could use something like a <![CDATA[]]> wrapper around the test_message text. If I do a 'pdbtool test --validate' against a database that includes the offending test_message I get an error, but if I wrap it in a CDATA it seems to run without error, although the parser doesn't actually seem to see that wrapped test message. I can also verify the test message with 'pdbtool match -M' command, so this isn't a big deal, just thought I'd bring attention to it and see if maybe I'm missing something or if this might be something to handle in a future release. John

1 0

syslog-ng wont start - please help
by vijay amruth 06 May '17

06 May '17

Hello Peter/ team, I am stuck with issue since four days, spent hours and hours, I am unable find a cause, I compiled syslog-ng from source and resolved all the dependencies, when I get to start it, it wont start. here is the out put, please have a look at it. Any help is appreciated. OS version : CentOS Linux release 7.3.1611 (Core) [root@my-host ~]# systemctl start syslog-ng.service Job for syslog-ng.service failed because a timeout was exceeded. See "systemctl status syslog-ng.service" and "journalctl -xe" for details. [root@my-host ~]# systemctl status syslog-ng.service ● syslog-ng.service - System Logger Daemon Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled) Active: activating (start) since Thu 2017-05-04 16:13:44 PDT; 29s ago Docs: man:syslog-ng(8) Main PID: 5089 (syslog-ng) CGroup: /system.slice/syslog-ng.service └─5089 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid May 04 16:13:44 my-host systemd[1]: Starting System Logger Daemon... [root@my-host ~]# journalctl -xe May 04 16:10:01 my-host systemd[1]: Starting Session 348 of user root. -- Subject: Unit session-348.scope has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit session-348.scope has begun starting up. May 04 16:10:21 my-host nslcd[1289]: [18832f] <group/member="root"> failed to bind to LDAP server ldap://ldap.vip.slc.e May 04 16:10:21 my-host nslcd[1289]: [18832f] <group/member="root"> no available LDAP server found: Can't contact LDAP May 04 16:10:21 my-host nslcd[1289]: [18832f] <group/member="root"> no available LDAP server found: Server is unavailab May 04 16:10:21 my-host CROND[5035]: (root) CMD (/usr/lib64/sa/sa1 1 1) May 04 16:10:23 my-host polkitd[676]: Registered Authentication Agent for unix-process:5039:17396299 (system bus name : May 04 16:10:23 my-host systemd[1]: Stopping Flush Journal to Persistent Storage... -- Subject: Unit systemd-journal-flush.service has begun shutting down -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit systemd-journal-flush.service has begun shutting down. May 04 16:10:23 my-host systemd-journal[2711]: Journal stopped -- Subject: The journal has been stopped -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- The system journal process has shut down and closed all currently -- active journal files. May 04 16:10:23 my-host systemd-journal[5046]: Runtime journal is using 16.0M (max allowed 391.1M, trying to leave 586. May 04 16:10:23 my-host systemd-journal[5046]: Runtime journal is using 16.0M (max allowed 391.1M, trying to leave 586. May 04 16:10:23 my-host systemd-journald[2711]: Received SIGTERM from PID 1 (systemd). May 04 16:10:23 my-host systemd-journal[5046]: Journal started -- Subject: The journal has been started -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- The system journal process has started up, opened the journal -- files for writing and is now ready to process requests. May 04 16:10:23 my-host polkitd[676]: Unregistered Authentication Agent for unix-process:5039:17396299 (system bus name May 04 16:10:23 my-host systemd[1]: Starting Flush Journal to Persistent Storage... -- Subject: Unit systemd-journal-flush.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit systemd-journal-flush.service has begun starting up. May 04 16:10:23 my-host systemd[1]: Started Flush Journal to Persistent Storage. -- Subject: Unit systemd-journal-flush.service has finished start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit systemd-journal-flush.service has finished starting up. -- -- The start-up result is done. May 04 16:12:14 my-host polkitd[676]: Registered Authentication Agent for unix-process:5066:17407401 (system bus name : May 04 16:12:14 my-host systemd[1]: Starting System Logger Daemon... -- Subject: Unit syslog-ng.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has begun starting up. May 04 16:13:08 my-host nslcd[1289]: [528108] <group/member="postfix"> failed to bind to LDAP server ldap://ldap.vip.sl May 04 16:13:08 my-host nslcd[1289]: [528108] <group/member="postfix"> no available LDAP server found: Can't contact LD May 04 16:13:08 my-host nslcd[1289]: [528108] <group/member="postfix"> no available LDAP server found: Server is unavai May 04 16:13:44 my-host systemd[1]: syslog-ng.service start operation timed out. Terminating. May 04 16:13:44 my-host systemd[1]: Failed to start System Logger Daemon. -- Subject: Unit syslog-ng.service has failed -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has failed. -- -- The result is failed. May 04 16:13:44 my-host systemd[1]: Unit syslog-ng.service entered failed state. May 04 16:13:44 my-host systemd[1]: syslog-ng.service failed. May 04 16:13:44 my-host polkitd[676]: Unregistered Authentication Agent for unix-process:5066:17407401 (system bus name May 04 16:13:44 my-host systemd[1]: syslog-ng.service holdoff time over, scheduling restart. May 04 16:13:44 my-host systemd[1]: Starting System Logger Daemon... -- Subject: Unit syslog-ng.service has begun start-up -- Defined-By: systemd -- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel -- -- Unit syslog-ng.service has begun starting up. [root@my-host ~]# ps -ef | grep syslog-ng root 5089 1 0 16:13 ? 00:00:00 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid root 5103 32598 0 16:14 pts/1 00:00:00 grep --color=auto syslog-ng [root@my-host ~]# systemctl status syslog-ng.service ● syslog-ng.service - System Logger Daemon Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; enabled; vendor preset: enabled) Active: activating (start) since Thu 2017-05-04 16:13:44 PDT; 1min 6s ago Docs: man:syslog-ng(8) Main PID: 5089 (syslog-ng) CGroup: /system.slice/syslog-ng.service └─5089 /usr/sbin/syslog-ng -F -p /var/run/syslogd.pid May 04 16:13:44 my-host systemd[1]: Starting System Logger Daemon... [root@my-host ~]#cat /usr/lib/systemd/system/syslog-ng.service [Unit] Description=System Logger Daemon Documentation=man:syslog-ng(8) [Service] Type=notify ExecStart=/usr/sbin/syslog-ng -F $SYSLOGNG_OPTS -p /var/run/syslogd.pid ExecReload=/bin/kill -HUP $MAINPID EnvironmentFile=-/etc/default/syslog-ng EnvironmentFile=-/etc/sysconfig/syslog-ng StandardOutput=journal StandardError=journal Restart=on-failure Environment=SYSTEMD_LOG_LEVEL=debug [Install] WantedBy=multi-user.target -- Thanks, Vijay Amrut.

2 5

PatternDB with CiscoSNMP Module
by Schoonover, Mark E HHHH 05 May '17

05 May '17

Thanks for reading, I'm in the process of creating a log to trap for Cisco devices. I have it working for facilities like this: TRINITY-2-TRINITY_SYSLOG_CRIT with this pattern: @ESTRING::%@@ESTRING:.cisco.Facility:-@@ESTRING:.cisco.Severity:-@@ESTRING:.cisco.MsgName::@ @ANYSTRING:.cisco.MsgText@ Now there are other facilities with an additional hyphen before the severity: EARL-DFC1-1-EXCESSIVE_PARITY_ERROR: Apr 26 14:04:45 hostname 565049: Apr 26 18:04:43.476 UTC: %EARL-DFC1-1-EXCESSIVE_PARITY_ERROR: EARL 0: Parity error detected in VRAM' which parses into: Testing message: program='EARL-DFC1-1-EXCESSIVE_PARITY_ERROR' message='Apr 26 14:04:45 hostname 565049: Apr 26 18:04:43.476 UTC: %EARL-DFC1-1-EXCESSIVE_PARITY_ERROR: EARL 0: Parity error detected in VRAM' Match name='.classifier.rule_id', value='09944c71-95eb-4bc0-8575-936931d85715', expected='09944c71-95eb-4bc0-8575-936931d85715' Wrong match name='.cisco.Facility', value='EARL', expected='EARL-DFC1' Wrong match name='.cisco.Severity', value='DFC1', expected='1' Wrong match name='.cisco.MsgName', value='1-EXCESSIVE_PARITY_ERROR', expected='EXCESSIVE_PARITY_ERROR' Match name='.cisco.MsgText', value='EARL 0: Parity error detected in VRAM', expected='EARL 0: Parity error detected in VRAM' I've tried a pattern that would parse EARL and DFC1 into separate variables, then append them into .cisco.Facility - which didn't work. Still that leaves the pattern hardcoded to just two hypens. I've also tried hard coding EARL-DFC1 into the pattern, then use value to set .cisco.Facility correctly. This approach works but being hardcoded, another Facility with embedded hyphens will fail. Ideally I'd like to have a pattern that matches on the last hyphen before the severity and capture that. I could parse using a regexp and setting $1, $2, $3, and $4 to the appropriate variables but I'm concerned about performance issues - I'm thinking this is a last resort solution. Any thoughts on how to proceed? Regards, Mark Schoonover - KA6WKE Infrastructure Engineering Manager, Splunk Architect ENE : Tools, Instrumentation and Common Services Team Office: 32.8697° N, 116.9711° W Phone : 770-261-7934 Email : mark.schoonover(a)cigna.com<mailto:mark.schoonover@cigna.com> HPSM Team: ENE NMS Engineering Confidential, unpublished property of CIGNA. Do not duplicate or distribute. Use and distribution limited solely to authorized personnel. ¬© Copyright 2017 CIGNA ------------------------------------------------------------------------------ CONFIDENTIALITY NOTICE: If you have received this email in error, please immediately notify the sender by e-mail at the address shown. This email transmission may contain confidential information. This information is intended only for the use of the individual(s) or entity to whom it is intended even if addressed incorrectly. Please delete it from your files if you are not the intended recipient. Thank you for your compliance. Copyright (c) 2017 Cigna ==============================================================================

4 5

unable to start syslog-ng.
by vijay amruth 01 May '17

01 May '17

Hello All, Hope you are all doing great. I am unable to start syslog-ng service Here is some information: *OS Version: Cent OS 7.3* *I compiled it from a tar ball..* [root@xxxxx system]# syslog-ng -V syslog-ng 3.9.1 Installer-Version: 3.9.1 Revision: Module-Directory: /usr/local/lib/syslog-ng Module-Path: /usr/local/lib/syslog-ng Available-Modules: kvformat,cef,disk-buffer,add-contextual-data,syslogformat,afsocket,affile,afprog,afuser,afamqp,afmongodb,csvparser,confgen,system-source,linux-kmsg-format,basicfuncs,cryptofuncs,dbparser,afstomp,pseudofile,graphite,sdjournal,date Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off *I manually added this file, this didn't come with install...* [root@xxxxx system]# cat syslog-ng.service [Unit] Description=System Logger Daemon Documentation=man:syslog-ng(8) [Service] Type=notify Sockets=syslog.socket ExecStart=/usr/sbin/syslog-ng -F -p /var/run/syslogd-ng.pid --fd-limit 50000 ExecReload=/bin/kill -HUP $MAINPID StandardOutput=null Restart=on-failure [Install] WantedBy=multi-user.target Alias=syslog.service [root@sl-sz3-splunk01 system]# pwd /lib/systemd/system *Had parser errors:* [root@xxxxxx ~]# syslog-ng -s Error parsing config, Error compiling template (Unknown template function "format-json") in /usr/local/share/syslog-ng/include/scl/cim/template.conf at line 23, column 32: included from /usr/local/etc/scl.conf line 29, column 1 included from /usr/local/etc/syslog-ng.conf line 8, column 1 template-function "format-cim" "$(format-json --pair @timestamp='${R_ISODATE}' --pair @message='${MSG}' --key .cim.* --shift 5 --key _* --key .* --replace-prefix .=_ --key *.*)\n"; ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng *Moved /usr/local/share/syslog-ng/include/scl/cim and no parser errors.* *But still unable to start.* [root@xxxxx system]# systemctl status syslog-ng.service ● syslog-ng.service - System Logger Daemon Loaded: loaded (/usr/lib/systemd/system/syslog-ng.service; disabled; vendor preset: enabled) Active: failed (Result: start-limit) since Mon 2017-05-01 10:46:58 PDT; 7s ago Docs: man:syslog-ng(8) Process: 2170 ExecStart=/usr/sbin/syslog-ng -F -p /var/run/syslogd-ng.pid --fd-limit 50000 (code=exited, status=203/EXEC) Main PID: 2170 (code=exited, status=203/EXEC) May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: syslog-ng.service: main process exited, code=exited, status=203/EXEC May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: Failed to start System Logger Daemon. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: Unit syslog-ng.service entered failed state. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: syslog-ng.service failed. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: syslog-ng.service holdoff time over, scheduling restart. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: start request repeated too quickly for syslog-ng.service May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: Failed to start System Logger Daemon. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: Unit syslog-ng.service entered failed state. May 01 10:46:58 sl-sz3-splunk01.slc.ebay.com systemd[1]: syslog-ng.service failed. What I am I missing? Any help is appreciated. Thank you. -- Thanks, Vijay Amrut.

3 3

syslog-ng 3.9.1 erroneously matching regex
by Soumyadip Das Mahapatra 01 May '17

01 May '17

Hi folks, I raised https://github.com/balabit/syslog-ng/issues/1455 Could you guys help me troubleshooting the issue. It looks to be a buggy regex match. Thanks, Soumyadip

2 1

bug in syslog-ng 3.9.1 triggered by bug in ivykis >= 0.40
by Lennert Buytenhek 29 Apr '17

29 Apr '17

Hello! (I subscribed to this mailing list to be able to post, but I have mailing list mail delivery disabled -- please CC me on replies.) A syslog-ng user tried to use syslog-ng with ivykis 0.41 and ran into the following issue: https://github.com/buytenh/ivykis/issues/11 In a nutshell, syslog-ng seems to be registering an ivykis timer (iv_timer) with timeout tv_sec = 0, tv_nsec = 0, and ivykis >= 0.40 added support for timerfd_create, but due to a bug in ivykis, this {0, 0} timeout would get passed into timerfd_settime() verbatim, and timerfd_settime() interprets this zero timeout as a request to disable the timerfd, which would cause timer processing to stop. I will release a new ivykis release with a workaround for this, but I figured that you'd probably want to know about this as well, as I don't think it makes sense for syslog-ng to be registering timers with an absolute timeout of zero. Cheers, Lennert

2 2

Syslog-ng and OSQUERY
by Dwijadas Dey 28 Apr '17

28 Apr '17

Hi List users Is it possible to send OSQUERY logs to syslog-ng 3.5 In the OSQUERY docs <https://osquery.readthedocs.io/en/latest/deployment/syslog/> rsyslog is configured to write logs to syslog. Does the same method applies to syslog-ng 3.5 ? Thanks and regards

4 16

Relay Server Config Help
by wiskbroom＠hotmail.com 27 Apr '17

27 Apr '17

Hello; I am running syslog-NG on a server inside of a DMZ, and on that server I'd like to just forward all messages into my internal syslog-NG server. I feel this is better than having to create a new firewall rule for each new DMZ node. I have a simple syslog-NG config that looks like this: ---------START syslog-ng.conf----------- @version:3.5 @include "scl.conf" @include "/etc/syslog-ng/conf.d/*.conf" options { time-reap(30); mark-freq(10); keep-hostname(yes); chain_hostnames (off); flush_lines (0); time_reopen (10); log_fifo_size (1000); use_dns (yes); use_fqdn (no); create_dirs (no); }; source s_relay { udp(ip(0.0.0.0) port(514) so_rcvbuf(425984)); tcp(ip(0.0.0.0) port(514) max-connections(250) so_rcvbuf(425984) log_iw_size(25000) so_keepalive(yes) log_fetch_limit(100)); syslog(ip(0.0.0.0) transport("tcp") port(1514) max-connections(500) log_iw_size(25000) flags("threaded") log_fetch_limit(100)); }; destination d_syslog_tcp { syslog("10.5.5.10" transport("tcp") port(514)); }; log { source(s_relay); destination(d_syslog_tcp); }; ----------END syslog-NG.conf------------- The problem that I am experiencing is that my messages, once received by my internal syslog-NG server, look like the following: Apr 26 17:31:06 relay-sng-server 126 <30>1 2017-04-26T17:32:01-04:00 relay-client-host-X appserv - - - 606881792 140565409392384 INFO: @2 SessionExchange::ProcessTCPRead Apr 26 17:31:06 relay-sng-server 124 <30>1 2017-04-26T17:32:01-04:00 relay-client-host-X appserv - - - 606881792 140565409392384 DEBUG: OpenSSLHandler::PerformHandshake I am hoping not to muck around too much with my config for my internal syslog-NG servers, and want for the messages above to appear as coming from relay-client-host-X and NOT relay-sng-server. It could potentially be nice to know that the message was first received by relay-sng-server, but... I was considering opening another TCP port on my internal syslog servers, and using that to send from DMZ, then a rewrite, but that all seems far more complicated than necessary. Am I missing something on my syslog-NG conf? Any suggestions greatly appreciated! Thanks all in advance, -Vadim

2 2

syslog 3.9 modifying ${MSG}?
by Nik Ambrosch 27 Apr '17

27 Apr '17

After moving from syslog-ng 3.5 to 3.9 i noticed that the contents of $PROGRAM and $MSG are being logged differently than before. Here is how they used to be logged: # program: 53 # message: Apr 19 09:35:35.713 GMT: %ILPOWER-5-POWER_GRANTED: Interface xxx: Power granted | That is the full message (as seen on the device) which is optimal behavior. Below is the behavior i’m seeing with syslog-ng 3.9 with similar configuration: # program: GMT # message: %ILPOWER-5-POWER_GRANTED: Interface xxx: Power granted The value of program and message are altered but everything else is the same. I’ve been investigating the date parser and the flags(no-parse) options but haven’t had any luck getting a properly formatted message yet. If anyone has any ideas on how to get the old behavior back it would be greatly appreciated.

3 5