June 2017 - syslog-ng - lists.balabit.hu

mark-mode
by Fabien Wernli 29 Jun '17

29 Jun '17

Hi, What would be the best way to identify a message originating from mark-mode? I don't see any obvious way to do so, apart from matching the message for '-- MARK --'. Here's an example on how it looks: { "YEAR_DAY": "180", "YEAR": "2017", "WEEK_DAY_NAME": "Thursday", "WEEK_DAY_ABBREV": "Thu", "WEEK_DAY": "5", "WEEKDAY": "Thu", "WEEK": "26", "USEC": "233311", "UNIXTIME": "1498744609", "TZOFFSET": "+02:00", "TZ": "+02:00", "TAGS": ".source.#anon-source0", "TAG": "0d", "S_YEAR_DAY": "180", "S_YEAR": "2017", "S_WEEK_DAY_NAME": "Thursday", "S_WEEK_DAY_ABBREV": "Thu", "S_WEEK_DAY": "5", "S_WEEKDAY": "Thu", "S_WEEK": "26", "S_USEC": "233311", "S_UNIXTIME": "1498744609", "S_TZOFFSET": "+02:00", "S_TZ": "+02:00", "S_STAMP": "Jun 29 15:56:49", "S_SEC": "49", "S_MSEC": "233", "S_MONTH_WEEK": "4", "S_MONTH_NAME": "June", "S_MONTH_ABBREV": "Jun", "S_MONTH": "06", "S_MIN": "56", "S_ISODATE": "2017-06-29T15:56:49+02:00", "S_HOUR12": "03", "S_HOUR": "15", "S_FULLDATE": "2017 Jun 29 15:56:49", "S_DAY": "29", "S_DATE": "Jun 29 15:56:49", "S_AMPM": "PM", "SYSUPTIME": "207", "STAMP": "Jun 29 15:56:49", "SOURCEIP": "127.0.0.1", "SOURCE": "#anon-source0", "SEC": "49", "R_YEAR_DAY": "180", "R_YEAR": "2017", "R_WEEK_DAY_NAME": "Thursday", "R_WEEK_DAY_ABBREV": "Thu", "R_WEEK_DAY": "5", "R_WEEKDAY": "Thu", "R_WEEK": "26", "R_USEC": "233311", "R_UNIXTIME": "1498744609", "R_TZOFFSET": "+02:00", "R_TZ": "+02:00", "R_STAMP": "Jun 29 15:56:49", "R_SEC": "49", "R_MSEC": "233", "R_MONTH_WEEK": "4", "R_MONTH_NAME": "June", "R_MONTH_ABBREV": "Jun", "R_MONTH": "06", "R_MIN": "56", "R_ISODATE": "2017-06-29T15:56:49+02:00", "R_HOUR12": "03", "R_HOUR": "15", "R_FULLDATE": "2017 Jun 29 15:56:49", "R_DAY": "29", "R_DATE": "Jun 29 15:56:49", "R_AMPM": "PM", "RUNID": "1", "PRIORITY": "notice", "PRI": "13", "MSG": "dl", "MSEC": "233", "MONTH_WEEK": "4", "MONTH_NAME": "June", "MONTH_ABBREV": "Jun", "MONTH": "06", "MIN": "56", "MESSAGE": "dl", "LOGHOST": "localhost.localdomain", "LEVEL_NUM": "5", "LEVEL": "notice", "ISODATE": "2017-06-29T15:56:49+02:00", "HOUR12": "03", "HOUR": "15", "HOST_FROM": "localhost", "HOSTID": "abb0b0e5", "HOST": "localhost", "FULLDATE": "2017 Jun 29 15:56:49", "FILE_NAME": "/dev/stdin", "FACILITY_NUM": "1", "FACILITY": "user", "DAY": "29", "DATE": "Jun 29 15:56:49", "C_YEAR_DAY": "180", "C_YEAR": "2017", "C_WEEK_DAY_NAME": "Thursday", "C_WEEK_DAY_ABBREV": "Thu", "C_WEEK_DAY": "5", "C_WEEKDAY": "Thu", "C_WEEK": "26", "C_UNIXTIME": "1498744609", "C_TZOFFSET": "-00:00", "C_TZ": "-00:00", "C_STAMP": "Jun 29 13:56:48", "C_SEC": "48", "C_MONTH_WEEK": "4", "C_MONTH_NAME": "June", "C_MONTH_ABBREV": "Jun", "C_MONTH": "06", "C_MIN": "56", "C_ISODATE": "2017-06-29T13:56:48-00:00", "C_HOUR": "13", "C_FULLDATE": "2017 Jun 29 13:56:48", "C_DAY": "29", "C_DATE": "Jun 29 13:56:48", "BSDTAG": "5B", "AMPM": "PM" }

1 0

Syslog-ng 3.9
by PENLAND,MATTHEW & DANA 28 Jun '17

28 Jun '17

I have several systems that are sending there log files to syslog-ng, without issue. I having an issue getting syslog-ng to work with logs from our Brocade MLX’s. I run wireshark on the syslog server and see the file hit it. However, syslog-ng does not put it in its folder or any where else. Sections of the config file relevant to the Brocade. destination d_bro { file("/home/hosts/brocade/$YEAR$MONTH$DAY-$HOST.log"); }; ##################### # BROCADE ##################### filter f_bro { not level(debug) and ( host( "BROCADE-BG-MLX" ) or host( "BROCADE-BG-FESX" ) or host( "BROCADE-BR-MLX" ) or host( "BROCADE-DUCK-FESX" ) or host( "BROCADE-ELJY-MLX" ) or host( "BROCADE-JSPR-MLX" ) or host( "BROCADE-STUDIO-FESX" ));}; log { source(s_net); filter(f_bro); destination(d_bro); }; Wireshark on the server - Capture.PNG

3 2

syslog-ng poll: containers
by Czanik, Péter 28 Jun '17

28 Jun '17

Hi, Yesterday I created another poll, this time about containers: https://twitter.com/PCzanik/status/879704447743533056 Please vote and also re-tweet it, so more people are reached. Thanks, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

1 0

java module crashes on recent kernels addressing CVE-2017-1000364
by Czanik, Péter 26 Jun '17

26 Jun '17

Hi, Due to an incompatible fix in recent kernels (to resolve CVE-2017-1000364), the Java module in syslog-ng crashes. This is used for example by the Elasticsearch destination. Right now the workaround is to use a kernel without the fix. Follow https://github.com/balabit/syslog-ng/issues/1562 to see any updates on the issue. Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

1 0

java destinations
by Czanik, Péter 26 Jun '17

26 Jun '17

Hi, I wanted to send some logs to Elasticsearch, but I did not succeed. Syslog-ng crashed. Actually it seems to be something JVM related. A bit too many things changed recently in my test environment: - a new laptop - a new version of vmware workstation - a new version of glibc and openjdk inside the vm-s (I always start with a security update...) The problem appeared both with syslog-ng 3.9 and 3.10 on both openSUSE Leap and CentOS 7. Have you ever seen anything similar? Bye, CzP This one is from CentOS7 where I already tried rolling back glibc and openjdk: [2017-06-23T13:25:22.816610] Compiling d_elastic reference [destination] at [/etc/syslog-ng/conf.d/es2.conf:27:3] [2017-06-23T13:25:22.816968] Seeking the journal to the start position; [2017-06-23T13:25:22.817808] Processing the time zone file (32bit part); filename='/usr/share/zoneinfo/UTC' # # A fatal error has been detected by the Java Runtime Environment: # # SIGBUS (0x7) at pc=0x00007f5ecc511c18, pid=1651, tid=0x00007f5ee3949b40 # # JRE version: (8.0_121-b13) (build ) # Java VM: OpenJDK 64-Bit Server VM (25.121-b13 mixed mode linux-amd64 compressed oops) # Problematic frame: # j java.lang.Object.<clinit>()V+0 # # Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again # # An error report file with more information is saved as: # /tmp/hs_err_pid1651.log # # If you would like to submit a bug report, please visit: # http://bugreport.java.com/bugreport/crash.jsp # Aborted (core dumped) This one is from openSUSE, where I had gdb installed... linux-pzl9:/var/lib/systemd/coredump # gdb -c core.syslog-ng.0.46a0d0e824ce4fa29a644ef713064044.2015.1498211596000000 /usr/sbin/syslog-ng GNU gdb (GDB; openSUSE Leap 42.2) 7.11.1 Copyright (C) 2016 Free Software Foundation, Inc. License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html > This is free software: you are free to change and redistribute it. There is NO WARRANTY, to the extent permitted by law. Type "show copying" and "show warranty" for details. This GDB was configured as "x86_64-suse-linux". Type "show configuration" for configuration details. For bug reporting instructions, please see: <http://bugs.opensuse.org/>. Find the GDB manual and other documentation resources online at: <http://www.gnu.org/software/gdb/documentation/>. For help, type "help". Type "apropos word" to search for commands related to "word"... Reading symbols from /usr/sbin/syslog-ng...(no debugging symbols found)...done. [New LWP 2015] [New LWP 2016] [New LWP 2017] [New LWP 2018] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib64/libthread_db.so.1". Core was generated by `syslog-ng -Fvde'. Program terminated with signal SIGSEGV, Segmentation fault. #0 _expand_stack_to (bottom=0x7ffe43f70fff "", bottom@entry=0x7ffe43f70990 "") at /usr/src/debug/icedtea-3.4.0/openjdk/hotspot/src/os/linux/vm/os_linux.cpp:673 673 /usr/src/debug/icedtea-3.4.0/openjdk/hotspot/src/os/linux/vm/os_linux.cpp: No such file or directory. [Current thread is 1 (Thread 0x7ff2ab992b40 (LWP 2015))] Missing separate debuginfos, use: zypper install syslog-ng-debuginfo-3.10.1-2.3.x86_64 (gdb) bt full #0 _expand_stack_to (bottom=0x7ffe43f70fff "", bottom@entry=0x7ffe43f70990 "") at /usr/src/debug/icedtea-3.4.0/openjdk/hotspot/src/os/linux/vm/os_linux.cpp:673 sp = 0x7ffe43f7a120 "" size = 37153 p = 0x7ffe43f70ff0 "" #1 0x00007ff2a5f6f184 in os::Linux::manually_expand_stack (t=t@entry=0x19e9000, addr=0x7ffe43f70990 "") at /usr/src/debug/icedtea-3.4.0/openjdk/hotspot/src/os/linux/vm/os_linux.cpp:686 mask_all = {__val = {18446744067267100671, 18446744073709551615 <repeats 15 times>}} old_sigset = {__val = {18446744067266838271, 140680142549761, 140730038723104, 140680138113208, 1140303776, 210453397503, 140730038723232, 140730038724512, 1, 27171840, 18446744073709551615, 4, 140730038723200, 140680140758410, 140730038723208, 18446744069414584320}} #2 0x00007ff2a5f77f7d in JVM_handle_linux_signal (sig=11, info=0x7ffe43f7a530, ucVoid=0x7ffe43f7a400, abort_if_unrecognized=1) at /usr/src/debug/icedtea-3.4.0/openjdk/hotspot/src/os_cpu/linux_x86/vm/os_linux_x86.cpp:330 addr = <optimized out> uc = 0x7ffe43f7a400 thread = 0x19e9000 stub = 0x0 newset = {__val = {0, 0, 18446744073709551615, 2, 27670976, 140679847562464, 3, 27171760, 27171728, 140679847562559, 140679847562515, 0, 140730038723488, 140680143714222, 3, 140730038724512}} err = {<StackObj> = {<No data fields>}, _id = 1, _message = 0x19e9c00 "", _detail_msg = 0xffffffffffffffff <error: Cannot access memory at address 0xffffffffffffffff>, _thread = 0x4, _pc = 0x7ffe43f7a740 "\320\v\177\212\362\177", _siginfo = 0x7ff2a60051f8 <SharedRuntime::generate_i2c2i_adapters(MacroAssembler*, int, int, BasicType const*, VMRegPair const*, AdapterFingerPrint*)+3464>, _context = 0x3, _filename = 0xffffffffffffffff <error: Cannot access memory at address 0xffffffffffffffff>, _lineno = -1, _current_step = 48, _current_step_info = 0x7ff2a65619f0 <vtable for Relocation+16> "\240栥\362\177", _verbose = 0, static first_error = 0x0, static first_error_tid = -1, static coredump_status = false, static coredump_message = '\000' <repeats 1999 times>, _size = 140680227341344, static out = {<outputStream> = {<ResourceObj> = {<No data fields>}, _vptr.outputStream = 0x7ff2a655fc50 <vtable for fdStream+16>, _indentation = 0, _width = 80, _position = 0, _newlines = 0, _precount = 0, _stamp = {_counter = 0}}, _fd = 1, _need_close = false}, static log = {<outputStream> = {<ResourceObj> = {<No data fields>}, _vptr.outputStream = 0x7ff2a655fc50 <vtable for fdStream+16>, _indentation = 0, _width = 80, _position = 0, _newlines = 0, _precount = 0, _stamp = {_counter = 0}}, _fd = -1, _need_close = false}} t = 0x19e9000 shm = {<StackObj> = {<No data fields>}, _thread = 0x19e9000} vmthread = 0x0 pc = <optimized out> #3 0x00007ff2a5f6e3d8 in signalHandler (sig=11, info=0x7ffe43f7a530, uc=0x7ffe43f7a400) at /usr/src/debug/icedtea-3.4.0/openjdk/hotspot/src/os/linux/vm/os_linux.cpp:4354 orig_errno = 2 #4 <signal handler called> No symbol table info available. #5 0x00007ff29453dc26 in ?? () No symbol table info available. #6 0x00007ffe43f7a990 in ?? () No symbol table info available. #7 0x00007ff28a7d0d50 in ?? () No symbol table info available. #8 0x00007ffe43f7a9d8 in ?? () No symbol table info available. #9 0x00007ff28a892ff8 in ?? () No symbol table info available. #10 0x0000000000000000 in ?? () No symbol table info available. (gdb) Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

5 9

linux kernel tuning
by Jim Hendrick 23 Jun '17

23 Jun '17

Hi all, What are your experiences with tuning the kernel for UDP, TCP, etc. How long does it take to see results from a single change? We are working issues with UDP drops, and have made fairly radical increases in buffer sizes with no apparent improvement, only to re-visit the system in a few days and seeing it cleaned up. "intuitively" I have always thought it should happen right away - but that is not what we are seeing here now. Thoughts? Thanks in advance! Jim

2 1

wildcard-file() source
by Czanik, Péter 22 Jun '17

22 Jun '17

Hi, Last week I ran a poll on twitter about the new features of syslog/ng 3.10 and the wildcard-file() source was voted the most. Starting with version 3.10, syslog-ng can collect messages from multiple text files. You do not have to specify file names one by one, just use a wildcard to select which files to read. This is especially useful when you do not know the file names by the time syslog-ng is started. This is often the case with web servers with multiple virtual hosts. From this blog you can learn how to get started using the wildcard-file() source. https://www.balabit.com/blog/reading-multiple-files-wildcard-file-source-sy… Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

1 0

question Syslog-NG Set Thread Name ?
by rmkml 20 Jun '17

20 Jun '17

Hello, First, Big Thx everyone for very good Syslog-NG product! During middle/high load, it's always interesting for known what Jobs load Syslog-NG process, but if I think correctly, top -H not show what Thread Name is used ? Do you known it's possible Syslog-NG supporting Set Thread Name like pthread_setname_np ? Best Regards @Rmkml

1 0

template() parameter for apache-accesslog-parser
by Czanik, Péter 20 Jun '17

20 Jun '17

Hi, I ran into a situation, that an apache access log message arrives as a field in a CSV file. Rigth now the apache-accesslog-parser is always run on the MESSAGE template, but in this case I'd need to run this parser on csv.apache instead of MESSAGE. What do you think about the following change? It would keep the default behavior but enable me to use a different a template. Bye, CzP linux-pzl9:/usr/share/syslog-ng/include/scl/apache # diff -u /root/syslog-ng/scl/apache/apache.conf apache.conf --- /root/syslog-ng/scl/apache/apache.conf 2017-06-19 13:47:20.044547152 +0200 +++ apache.conf 2017-06-20 16:51:06.740528980 +0200 @@ -19,7 +19,7 @@ # COPYING for details. # ############################################################################# -block parser apache-accesslog-parser(prefix(".apache.")) { +block parser apache-accesslog-parser(prefix(".apache.") template("${MESSAGE}")) { channel { @@ -29,6 +29,7 @@ dialect(escape-double-char) flags(strip-whitespace) delimiters(" ") + template(`template`) quote-pairs('""[]') # field names match of that of Logstash columns("clientip", "ident", "auth", Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

2 1

tags not working
by Sandbox 19 Jun '17

19 Jun '17

Hi, I would like to transfer/store apache logs, but for some reason i cant tag them: I tried: source s_access { pipe("/var/log/apache2/pipe_access.log" tags("testtag"));. }; or source s_access { pipe("/var/log/apache2/pipe_access.log" program-override("testtag"));. }; Because it doesn't work with 'tags' i used the apache to format the message, i put the choosen word to the log format so it stores the correct place the log on the client and also on the server. Part of the log: - [19/Jun/2017:13:46:29 +0200] acc001 127.0.0.1 - - "GET /server-status?auto HTTP/1.1" 200 1781 If it possible i want to use 'tags' instead of manipulating apache logformats. I tried to read the pipes directly, but the above settings doesnt show, just the above log. I tried to tcpdump the traffic from client -> server (client side) but doesn't show any of the settings. It looks like for me, syslog-ng wont tag my logs. And i tried to read the stored log with file() and tag it, that doesn't work neither. What i am missing? Thanks, Robert

2 4