- syslog-ng - lists.balabit.hu

How to export syslogng to elasticsearch
by Mik J 19 May '17

19 May '17

Hello, I've read many documentation about how to export syslogng to elasticsearch but still don't get it # syslog-ng -V syslog-ng 3.7.2 Installer-Version: 3.7.2 Revision: Compile-Date: May 8 2017 10:54:55 Available-Modules: afprog,afsocket,afsql,afuser,basicfuncs,confgen,csvparser,dbparser,graphite,json-plugin,kvformat,linux-kmsg-format,pseudofile,syslogformat,system-source,affile,cryptofuncs Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off In syslog-ng.conf I need to add: @module mod-java @include "scl.conf" In scl.conf I have (I kept the default configuration) @define scl-root "`syslog-ng-data`/include/scl" @define include-path "`include-path`:`syslog-ng-data`/include" @include 'scl/*/*.conf' I don't understand really what should I configure. I also downloaded gradle # ls /usr/local/gradle-3.5 And Java # /usr/local/jdk-1.8.0/bin/java -version openjdk version "1.8.0_121" OpenJDK Runtime Environment (build 1.8.0_121-b13) OpenJDK 64-Bit Server VM (build 25.121-b13, mixed mode) Back to syslog-ng.conf I added # Test Elasticsearch filter f_MyTest { host("1.1.1.1"); }; log { source(s_net); filter(f_MyTest); destination(d_MyTest); }; destination d_elastic { elasticsearch( index("syslog-ng_${YEAR}.${MONTH}.${DAY}") type("test") cluster("syslog-ng") client_mode("transport") custom_id("${UNIQID}") flush-limit("10000") ); }; log { source(s_net); filter(f_MyTest); destination(d_elastic); flags(flow-control); }; But starting the daemon gives me this error # syslog-ng [2017-05-18T16:43:42.465496] Plugin module not found in 'module-path'; module-path='/usr/local/lib/syslog-ng', module='mod-java' Error parsing destination, destination plugin elasticsearch not found in /etc/syslog-ng/syslog-ng.conf at line 171, column 2: elasticsearch( ^^^^^^^^^^^^^

3 6

syslog-ng with SELinux in enforcing mode
by Czanik, Péter 19 May '17

19 May '17

Hi, Recently a large part of syslog-ng problem reports were related to SELinux not allowing access to some custom network ports or directories. You can read the basics of customizing SELinux (and Firewalld) rules for your environment in my latest blog: https://www.balabit.com/blog/using-syslog-ng-with-selinux-in-enforcing-mode/ Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

1 0

use of Syslog-ng to filter logs for forwarding to Splunk
by Stuart Martin 18 May '17

18 May '17

Hi We are looking to collect logs from servers and devices in our DMZ and then filter the logs of unnecessary information to then be sent to our internal Splunk instance. Would Syslog-ng OSE edition be capable of this task? I've started to read through your excellent documentation for the OSE edition, I was wondering what route I should take to configure it for the scenario described above? The link I am thinking is correct is the "To configure syslog-ng OSE as a relay that receives log messages from client hosts and forwards them to a central logserver, see Procedure 4.2, Configuring syslog-ng on server hosts<https://www.balabit.com/sites/default/files/documents/syslog-ng-ose-latest-…>." Does that sound correct? Many Thanks Kind Regards Stuart Martin Infrastructure Security Engineer Surrey Satellite Technology Ltd Tycho House, 20 Stephenson Road, Surrey Research Park, Guildford, GU2 7YE Tel: +44 (0)1483 803803 | Fax: +44 (0)1483 803804 | Email: s.martin(a)sstl.co.uk<mailto:s.martin@sstl.co.uk> www.sstl.co.uk<http://www.sstl.co.uk/> http://twitter.com/SurreySat P Please consider the environment before printing this e-mail This e-mail and any attachments may contain confidential and privileged information. If you are not the intended recipient, please notify the sender immediately by return e-mail, delete this e-mail and destroy any copies. Any dissemination or use of this information by a person other than the intended recipient is unauthorized and may be illegal. The information contained in this message is intended for the addressee only and may contain sensitive information. If you are not the addressee, please delete this message and notify the sender; you should not copy or distribute this message or disclose its contents to anyone. Any views or opinions expressed in this message are those of the individual(s) and not necessarily of the organisation. No reliance may be placed on this message without written confirmation from an authorised representative of its contents. No guarantee is implied that this message or any attachment is virus free or has not been intercepted and amended.

3 5

DMZ Relay Config (multiple ports)
by wiskbroom＠hotmail.com 18 May '17

18 May '17

Hello! I am trying to build a syslog-ng.conf for a DMZ relay that will listen on several TCP and UDP ports, all working just great. My config essentially captures traffic, then redirects each connection internally from itself. What happens if my internal host goes down? I am hoping to store locally if connectivity is down, but not sure where or how that would get defined. I need to keep a direct relationship between ports, as my internal syslog-ng treats these ports differently, so redirecting to the same as received is key. Below is my DMZ relay syslog-NG config file, any offers to make it better greatly appreciated. -Vadim -------------------------syslog-ng.conf------------------------------ @version:3.5 @include "scl.conf" @include "/etc/syslog-ng/conf.d/*.conf" options { time-reap(30); mark-freq(10); keep-hostname(yes); chain_hostnames (off); flush_lines (0); time_reopen (10); log_fifo_size (1000); use_dns (yes); use_fqdn (no); create_dirs (no); }; source s_udp-relay { udp(ip(0.0.0.0) port(514) so_rcvbuf(425984)); }; source s_tcp-relay-514 { tcp(ip(0.0.0.0) port(514) max-connections(250) so_rcvbuf(425984) log_iw_size(25000) so_keepalive(yes) log_fetch_limit(100)); }; source s_tcp-relay-7514 { tcp(ip(0.0.0.0) port(7514) max-connections(250) so_rcvbuf(425984) log_iw_size(25000) so_keepalive(yes) log_fetch_limit(100)); }; destination d_syslog_udp { udp("10.5.5.5" port(514)); }; destination d_syslog_tcp-514 { tcp("10.5.5.5" port(514)); }; destination d_syslog_tcp-7514 { tcp("10.5.5.5" port(7514)); }; log { source(s_udp-relay); destination(d_syslog_udp); }; log { source(s_tcp-relay-514); destination(d_syslog_tcp-514); }; log { source(s_tcp-relay-7514); destination(d_syslog_tcp-7514); };

3 2

syslog-ng-3.9.1-1 is crashing
by Harish Shetty 17 May '17

17 May '17

Hi Peter I have deployed syslog-ng-3.9.1-1.el6.x86_64 , where I am seeing one issue. Where there is disk usage in greater than 90%, It coredumps. Instead of that shouldnt it be handle the syslog-ng service gracefully? I have pasted part of core here and attached the coredump full file gdb) bt full #0 0x0000003b52a325e5 in raise () from /lib64/libc.so.6 No symbol table info available. #1 0x0000003b52a33dc5 in abort () from /lib64/libc.so.6 No symbol table info available. #2 0x0000003206c7506b in iv_fatal (fmt=<value optimized out>) at iv_fatal.c:57 ap = {{gp_offset = 8, fp_offset = 48, overflow_arg_area = 0x7f35c5aadac0, reg_save_area = 0x7f35c5aada00}} msg = "iv_init: can't initialize event dispatcher\000\006\062\000\ 000\000\177\000\000\bZ\003\000\000\000\000`KL\234\065\177\ 000\000?M006\062\000\000\000@֪\177\000\000@֪\177\000\000\ 177\000\000\b\000\000\000\000\000\000\000L\234\065\177\000\ 000L\234\065\177\000\000\000KL\234\065\177\000\000Ye006\062\000\000\000\001\ 000\000\000\000\000\000\000ت\177\000\000ت\177\000\000\177\000\000ت\177", '\000' <repeats 46 times>, "\004", '\000' <repeats 35 times>... #3 0x0000003206c76f2a in iv_fd_init (st=0x7f359c945850) at iv_fd.c:129 No locals. #4 0x0000003206c77499 in iv_init () at iv_main_posix.c:74 st = 0x7f359c945850 #5 0x0000003206c75b8e in iv_work_thread (_thr=0x996c540) at iv_work.c:177 thr = 0x996c540 pool = 0x1df6820 #6 0x0000003206c77def in iv_thread_handler (_thr=0x52933d0) at iv_thread_posix.c:142 __clframe = {__cancel_routine = 0x3206c77e50 <iv_thread_cleanup_handler>, __cancel_arg = 0x52933d0, __do_it = 1, __cancel_type = 0} thr = 0x52933d0 #7 0x0000003b53207aa1 in start_thread () from /lib64/libpthread.so.0 No symbol table info available. ---Type <return> to continue, or q <return> to quit--- #8 0x0000003b52ae8aad in clone () from /lib64/libc.so.6 No symbol table info available. Can you please check and let me know how can we fix this issue? Regards Harish shetty

4 9

Re: [syslog-ng] Syslog-ng input for beats ?
by Scot 17 May '17

17 May '17

I tried reading the mapping documentation in the Balabit Latest docs but I'm not connecting these dots, sorry. Can I use the templates for creating the ES index as a reference for json mapping in syslog-ng ? For example the attached winlogbeat.template.json which created the ES index.

2 3

Running into Error: Could not find Java SE Runtime Environment
by vijay amruth 17 May '17

17 May '17

Hello Peter/team/everyone, Post configure syslog-ng 3.9.1 on redhat 7.3, I am running to this error when running make, please look into this, any help is appreciated. GEN modules/java/syslog-ng-core/classes/org/syslog_ng/LogMessage.class Error: could not find libjava.so Error: Could not find Java SE Runtime Environment. make: *** [modules/java/syslog-ng-core/classes/org/syslog_ng/LogMessage.class] Error 2 Tried fixing path, didn't help...... [root@sml-collector02 syslog-ng-3.9.1]# export JRE_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.102-4.b14.el7.x86_64/jre [root@sml-collector02 syslog-ng-3.9.1]# export PATH=$PATH:$JRE_HOME/bin [root@sml-collector02 syslog-ng-3.9.1]# export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.102-4.b14.el7.x86_64 [root@sml-collector02 syslog-ng-3.9.1]# export JAVA_PATH=$JAVA_HOME [root@sml-collector02 syslog-ng-3.9.1]# export PATH=$PATH:$JAVA_HOME/bin Please look into it. Thank you. -- Thanks, Vijay Amrut.

1 0

in-list filter for substrings
by Clayton Dukes 16 May '17

16 May '17

Question: Is there are way to read in a file to match for substrings? Example: filter f_malware { in-list("/etc/syslog-ng/conf.d/malware.list", value("MSG")); }; rewrite rw_malware { set("Malware", value("PROGRAM") condition(filter(f_malware))); }; Then, given a string like: Dec 15 17:57:02 230.16.190.121 %ASA-5-304001: 67.16.126.205 Accessed URL 114.4.201.244:http://wvqwdhimen.jaqwejggvwxyangoqepw.tk/storageframe.html It would look at malware.list and see a line containing: wvqwdhimen.jaqwejggvwxyangoqepw.tk and set the Program name as "Malware" I see in the docs, it says ' Only exact matches are supported, partial and substring matches are not' so I don't think this will work. Is there a way to do anything like this? - Clayton

1 0

How to get list of IPs
by Sathish Sundaravel 13 May '17

13 May '17

Hi All I have syslog Ng server running on Solaris platform and configured to receive or accept events from all unix servers in one single log file. If I want to collect list of servers reporting to the log server , I have to manually extract list from a single log file which is too much time consuming . What would the best way ? Regards Sathish

5 6

PLEASE REMOVE MY EMAIL FROM THIS LIST...
by Alex Norton 13 May '17

13 May '17

THANKS ALEX

2 1