- syslog-ng - lists.balabit.hu

syslog-ng, json and handle escaping properly
by Arthur Lambert 24 Apr '20

24 Apr '20

Hello, I tried to migrate from syslog to Syslog-ng. I am trying to deploy it on an embedded target with buildroot. At the same time, I also tried to switch log format to JSON. In my c code, I use Syslog function and init with the openlog function: openlog("middleware", 0, LOG_LOCAL1); I have some macro to generate the json logs: #define _PRINT_DEBUG(M, ...) \ do { \ syslog(LOG_DEBUG, "\"category\":\"%s\",\"level\":\"DEBUG\",\"message\":\"" M "%s\"}\n", __DIR__, __VA_ARGS__); \ } while (0) I define a syslog-ng conf file: @version: 3.2X filter f_middleware { facility("local1"); }; template t_json_filetemplate { template("{\"timestamp\":\"${ISODATE}\",${MESSAGE}\n"); }; source s_kernel { file("/proc/kmsg" program_override("kernel")); }; source s_middleware { unix-stream("/dev/log"); }; destination d_kernel { file("/data/logs/kern.log"); }; destination d_middleware { file("/data/logs/middleware.log", template(t_json_filetemplate)); }; log { source(s_kernel); destination(d_kernel); }; log { source(s_middleware); filter(f_middleware); destination(d_middleware); }; So basically I want to redirect all my log from my code in /data/logs/middleware.log. I also want to redirect all kernel logs in /data/logs/kern.log I have two issues. The first one is escaping. If I want to print a JSON file the double quote will be not escaped. My first solution was to use escape feature from Syslog-ng: destination d_middleware { file("/data/logs/middleware.log", template("{\"timestamp\":\"${ISODATE}\",${MESSAGE}\n") template-escape(yes)); }; Problem is that now everything is escaped and not only the message, some example: {"timestamp":"2020-04-21T09:24:02+02:00",\"category\":\"BOOT\",\"level\":\"INFO\",\"message\":\"Check pstore partition..\"} {"timestamp":"2020-04-21T09:24:02+02:00",\"category\":\"BOOT\",\"level\":\"INFO\",\"message\":\"Start hardware watchdog daemon...\"} {"timestamp":"2020-04-21T09:24:02+02:00",\"category\":\"BOOT\",\"level\":\"INFO\",\"message\":\"Prepare asound configuration...\"} {"timestamp":"2020-04-21T09:24:02+02:00",\"category\":\"BOOT\",\"level\":\"INFO\",\"message\":\"Middleware is about to start...\"} {"timestamp":"2020-04-21T09:24:03+02:00",\"category\":\"BOOT\",\"level\":\"INFO\",\"message\":\"Load sdma firmware..\"} {"timestamp":"2020-04-21T09:24:03+02:00",\"category\":\"BOOT\",\"level\":\"INFO\",\"message\":\"Watchdog is about to start...\"} {"timestamp":"2020-04-21T09:24:03+02:00",\"category\":\"BOOT\",\"level\":\"INFO\",\"message\":\"Start mpu watchdog..\"} As you can see, everything is escaped because for Syslog-ng, everything is the message and not only the final part. I feel like I am completely using Syslog-ng in the wrong way right? How can I fix that? Do I have to handle escaping by myself manually in my middleware? Do I have to pre-process all logs manually by myself? It seems quite awful. I have a second issue which is probably related? When I tried to print the content of a file, the last '}' is truncated. The reason seems to be related to the presence of \n. I can reproduce the issue by just calling my PRINT_DEBUG macro as follow: PRINT_DEBUG ("YO YO YO KO\n"); PRINT_DEBUG ("YO YO YO OK"); The result: {"timestamp":"2020-04-21T09:24:03+02:00",\"category\":\"MPU_CORE\",\"level\":\"DEBUG\",\"message\":\"YO YO YO KO {"timestamp":"2020-04-21T09:24:03+02:00",\"category\":\"MPU_CORE\",\"level\":\"DEBUG\",\"message\":\"YO YO YO OK\"} My syslog version/module information : # syslog-ng -V -s syslog-ng 3 (3.26.1) Config version: 3.22 Installer-Version: 3.26.1 Revision: Compile-Date: Apr 10 2020 16:41:46 Module-Directory: /usr/lib/syslog-ng Module-Path: /usr/lib/syslog-ng Include-Path: /usr/share/syslog-ng/include Available-Modules: add-contextual-data,affile,afprog,afsocket,afstomp,afuser,appmodel,azure-auth-header,basicfuncs,cef,confgen,cryptofuncs,csvparser,dbparser,disk-buffer,examples,graphite,hook-commands,ht tp,kvformat,linux-kmsg-format,map-value-pairs,mod-python,pseudofile,snmptrapd-parser,stardate,syslogformat,system-source,tags-parser,tfgetent,timestamp,xml Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off Enable-Systemd: off -- - Arthur LAMBERT

3 2

syslog-ng-ctl: EOF occurred while reading control socket
by Faine, Mark R. (MSFC-IS40)[NICS] 24 Apr '20

24 Apr '20

I am seeing the message "EOF occurred while reading control socket" intermittently when using the syslog-ng-ctl command. Is there something misconfigured or anything I can do to prevent this? Thanks, -Mark

2 1

mystique with spoof_address
by Edvinas Kairys 16 Apr '20

16 Apr '20

Hello, i installed (yum install) following version on Centos 7 box. syslog-ng 3.5.6 Installer-Version: 3.5.6 Revision: Compile-Date: Dec 30 2015 19:57:24 Available-Modules: affile,afprog,afsocket-notls,afsocket-tls,afsocket,afstomp,afuser,basicfuncs,confgen,cryptofuncs,csvparser,dbparser,linux-kmsg-format,syslogformat,system-source Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-Linux-Caps: on Enable-Pcre: on My goal is to forward syslog messages 'untouched' but to change the source address to original one. For that case i'm using spoof-address. My conf is like this: options { flush_lines (0); time_reopen (10); log_fifo_size (1000); chain_hostnames (off); use_dns (no); use_fqdn (no); create_dirs (no); keep_hostname (no); mark-freq (0); }; source s_network { udp(ip(0.0.0.0) port(514) flags(no-parse)); }; destination d_syslog_tcp { network("10.13.33.125" transport("udp") port(5140) spoof-source(yes)); }; log { source(s_network); destination(d_syslog_tcp); }; log { source(s_network); filter(f_default); destination(d_mesg); }; # Source additional configuration files (.conf extension only) @include "/etc/syslog-ng/conf.d/*.conf" Strange thing, that when I enable spoof-source, some packets are not transmitted to the destination. Even TCPDUMP says that it's sent, but i dont see some logs on destination box. Could it be something with spoof_source command ? Also i didn't compiled it because i saw that SPOOF functionality is on in syslog-ng -V output. Any suggestions ? Thanks

2 4

issues with rewrite and subst
by Russell Fulton 14 Apr '20

14 Apr '20

version 3.26.1 I am attempting to use rewrite to copy part of one macro into a new one: rewrite r_user1 { set( "${user}", value("user_name"));}; rewrite r_user2 { subst( "^.+$(.+)$", "${1}", value("user_name")) ;}; The first simply copies user into user_name. The second is intended to extract anything in parentheses and throw away the rest. here is a cleaned up dump of json output including the macros from the regexp: {"user_name":")","user”:”yyyyn@gmail.com(xxx549)","2":")","1":")","0”:”yyyy@gmail.com(xxx549)”} I believe ‘1’ should contain ‘xxx549’ not ‘)’. I have tested the regexpn in ruby’s irb and it works fine there. here is another example this one where the regexp should not have matched at all: {"user_name":"9","user":"ping999","2":"9","1":"9","0":"ping999”} macros 1 and 2 always seem to contain the last character of the original string.

2 1

Regular occurrences of Error writing control channel; error='Broken pipe (32)'
by Ahmed Soyef 09 Apr '20

09 Apr '20

Hi All, We've been running syslog-ng and have recently started to see the above message in our logs. Sometimes running syslog-ng-ctl stats takes very long and results in similar error messages. Does anyone know why this may be the case? Do these errors cause any issues with log flows through the syslog-ng service? Do they cause issues with recording of stats? Kind regards, Soyef ********************************************************************** This email and any files transmitted with it are private and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please return it to the address it came from telling them it is not for you and then delete it from your system. This email message has been swept for computer viruses. **********************************************************************

3 3

Insider 2020-04: syslog-ng RPM; mult-line-timeout; FreeBSD;
by Peter Czanik (pczanik) 09 Apr '20

09 Apr '20

Dear syslog-ng users, This is the 80th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. In these difficult times, when many of us stay at home, we can decide what to do with that bit of extra time that we don’t spend commuting. Some of us turn to entertainment or hobbies, do long-neglected things around the house, or even focus on studying and honing skills. If you ran out of movies to watch and want to spend your time on something more practical, this might be the best time to learn something new about syslog-ng. One starting point for that is the reference documentation, available for each release separately: https://www.syslog-ng.com/technical-documents/list/syslog-ng-open-source-ed… Another possibility is to check out our ever-growing collection of syslog-ng blogs. These describe a large number of different syslog-ng use cases. Blogs are available at https://www.syslog-ng.com/community/b/blog If you want to deep dive into configuring syslog-ng, here are some of the highlights: * The building blocks of syslog-ng: https://www.syslog-ng.com/community/b/blog/posts/building-blocks-of-syslog-… * Calculate PI with syslog-ng: https://www.syslog-ng.com/community/b/blog/posts/calculate-pi-syslog-ng * Creating heat maps using the new syslog-ng geoip2 parser: https://www.syslog-ng.com/community/b/blog/posts/creating-heat-maps-using-n… * Extend syslog-ng in Python: https://www.syslog-ng.com/community/tags/python Even if it is often not real-time, you can also chat with syslog-ng users and developers. The syslog-ng Gitter channel is a place to ask syslog-ng-related questions, but you are also very welcome to share your experiences about syslog-ng here. A success story not just feels good for developers but can also inspire other users. It is available at https://gitter.im/syslog-ng/syslog-ng NEWS Overview of syslog-ng RPM repositories -------------------------------------- This blog provides you an overview of syslog-ng RPM repositories: why to have unofficial packages at all and what are the advantages and disadvantages of the different unofficial repositories. https://www.syslog-ng.com/community/b/blog/posts/overview-of-syslog-ng-rpm-… Multi-line-timeout: making sure your last multi-line message is not lost -------------------------------- When your application has a problem that it cannot handle, then Java, PHP and other environments often write multi-line error messages. These long messages include information that is useful for developers, but it might be difficult to handle for logging systems. A multi-line log message is usually saved when the beginning of the next message is detected, which can make handling the last message problematic. This is where multi-line-timeout can help. After a pre-configured timeout, the last message is saved by syslog-ng. Losing a message when there are thousands of them is probably not a big deal. But if you are lucky, the error messages are rare. Maybe just a single message in a file. In either case, you can lose most or even all messages. If you do not want to lose these messages, configure multi-line-timeout. https://www.syslog-ng.com/community/b/blog/posts/multi-line-timeout-making-… syslog-ng in FreeBSD ports -------------------------- Recently we explored the different syslog-ng packages available for RPM-based Linux distributions, used by the majority of our Linux users. This time we focus on FreeBSD, the platform of choice for most of our non-Linux users. >From this blog, you can learn about the features that are in the binary syslog-ng packages, how to compile syslog-ng for yourself from ports and the current syslog-ng version policy in ports. NEW WHITE PAPERS * Upgrade to State-of-the-Art Log Management: https://www.syslog-ng.com/whitepaper/log-management-essentials8137754/ WEBINARS * Learn how to easily integrate Azure Sentinel into your log-data strategy: https://www.syslog-ng.com/webcast-ondemand/learn-how-to-easily-integrate-az… * You can browse recordings of past webinars at https://www.syslog-ng.com/events/ Your feedback and news, or tips about the next issue are welcome. To read this newsletter online, visit: https://syslog-ng.com/blog/ Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

can not load geoip2 plugin
by Russell Fulton 09 Apr '20

09 Apr '20

HI This is a locally compiled application (version 3.26.1) and I am trying to get the geoip2 stuff working. I had similar problems a week or two back with elasticsearch_http destination plugin. This seems like it might be easier to diagnose since the error says where it is looking for the plugin. rful011@secmgrprd02:~$ /usr/local/syslog-ng-3.26.1/sbin/syslog-ng -s -f /etc/syslog-ng/geoip.conf [2020-04-08T07:30:19.551845] Plugin module not found in 'module-path'; module-path='/usr/local/syslog-ng-3.26.1/lib/syslog-ng', module='geoip2' Error parsing config, syntax error, unexpected LL_ERROR, expecting $end in /etc/syslog-ng/geoip.conf:4:1-5:1: 1 @version: 3.21 2 3 @include "scl.conf" 4-----> @module geoip2 4-----> ^^^^^^^^^^^^^^^ 5-----> # 6 7 options { 8 use_dns (no); 9 use_fqdn (no); rful011@secmgrprd02:~$ ls -l /usr/local/syslog-ng-3.26.1/lib/syslog-ng/libgeoip2-plugin.* -rwxr-xr-x 1 root root 1271 Mar 31 08:57 /usr/local/syslog-ng-3.26.1/lib/syslog-ng/libgeoip2-plugin.la -rwxr-xr-x 1 root root 158912 Mar 31 08:57 /usr/local/syslog-ng-3.26.1/lib/syslog-ng/libgeoip2-plugin.so any ideas why it is not loading the library?

2 2

3.26 RHEL/CentOS 6 packages
by Peter Czanik (pczanik) 02 Apr '20

02 Apr '20

Hi, syslog-ng 3.26 packages are now also available for RHEL/CentOS 6: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng326epel6/ And if you use syslog-ng on RHEL/CentOS I'd appreciate if you could answer a quick poll on Twitter: https://twitter.com/PCzanik/status/1245692669889777667 Bye, Peter Czanik (CzP) <peter.czanik(a)oneidentity.com> Balabit (a OneIdentity company) / syslog-ng upstream https://syslog-ng.com/community/ https://twitter.com/PCzanik

1 0

errors with elasticsearch_http on 2.21 & 3.22
by Russell Fulton 02 Apr '20

02 Apr '20

I have had this config running happily for several months and today I tried to add the geoip2 plugin and managed to break everything. I first realised that I needed to recompile syslog-ng to get the geoip2 plugin but forgot I was running a locally compiled version . I tried apt install syslog-ng-mod-geoip and this broke things badly and I have been trying to recover for the last few hours :( The issue is that any attempt to start syslog-ng with the original config that includes a destination of elasticsearch_http get an error: Error parsing destination statement, destination plugin elasticsearch_http not found in /usr/local/syslog-ng-3.22/etc/syslog-ng.conf:33:3-33:21 Judging from what I found on th ’Net this is what one would expect if the syslog-ng version was prior to 3.21 and did not support the http dest. I first tried recompiling and installing 3.21 (in a different dir) with geoip2 enabled. That went with out errors but still would not process my config. Aside: At this point I found that in spite of having the geoip2 module included, syslog-ng failed to find it when I copied the setup from the manual. long story short the manual says the module is ‘geoip2’ when in fact it is ‘geoip2-plugin’ and that is the name you have to use in the @module. Weird! I then compiled 3.22 since I had the tar file and this behaved the same. rful011@secmgrprd02:~$ cat short.conf @version: 3.21 @include "scl.conf" @module geoip2-plugin source s_loghost { network( transport("tcp") flags(no-multi-line) port(1514) keep-alive(yes)); }; destination d_elastic { elasticsearch_http( index("auth_${YEAR}.${MONTH}.${DAY}") type("auth") persist-name("auth") template( "$(format-json --scope rfc3164 --scope nv-pairs --exclude R_DATE --key ISODATE)\n") url("http://secesprd01.its.auckland.ac.nz:9200/_bulk") ); }; log { source(s_loghost); destination(d_elastic ); flags(flow-control); }; rful011@secmgrprd02:~$ /usr/local/syslog-ng-3.21/sbin/syslog-ng -s -c ~/short.conf Error parsing destination statement, destination plugin elasticsearch_http not found in /usr/local/syslog-ng-3.21/etc/syslog-ng.conf:33:3-33:21: 28 file( "/home/rful011/test.log" template(t_db_parsed_test)); 29 }; 30 31 32 destination d_elastic { 33----> elasticsearch_http( 33----> ^^^^^^^^^^^^^^^^^^ 34 index("auth_${YEAR}.${MONTH}.${DAY}") 35 type("auth") 36 persist-name("auth") 37 template( "$(format-json --scope rfc3164 --scope nv-pairs --exclude R_DATE --key ISODATE)\n") 38 url("http://secesprd01.its.auckland.ac.nz:9200/_bulk”) rful011@secmgrprd02:~$ /usr/local/syslog-ng-3.21/sbin/syslog-ng --version syslog-ng 3 (3.21.1) Config version: 3.21 Installer-Version: 3.21.1 Revision: Compile-Date: Mar 29 2020 11:57:39 Module-Directory: /usr/local/syslog-ng-3.21/lib/syslog-ng Module-Path: /usr/local/syslog-ng-3.21/lib/syslog-ng Include-Path: /usr/local/syslog-ng-3.21/share/syslog-ng/include Available-Modules: add-contextual-data,affile,afprog,afsocket,afstomp,afuser,appmodel,basicfuncs,cef,confgen,cryptofuncs,csvparser,date,dbparser,disk-buffer,examples,geoip2-plugin,tfgetent,graphite,hook-commands,http,json-plugin,kvformat,linux-kmsg-format,map-value-pairs,pseudofile,mod-python,snmptrapd-parser,stardate,syslogformat,system-source,tags-parser,xml Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off Enable-Systemd: off

3 8

Syslog-NG Mysql Parsing to separate column
by Sync IT 01 Apr '20

01 Apr '20

Hi I am new to syslog-ng. I am receiving my logs perfectly to mysql. now i want to separate the message log into different column in mqsql. Currently i am receiving messages like this R2_DST_RTR forward: in:<pppoe-user1> out:vlan100-Default-Internet, src-mac 11:22:33:44:55:66, proto TCP (ACK,FIN), 172.16.41.159:42670->23.46.16.231:80, NAT (172.16.41.159:42670->9.9.9.9:42670)->23.46.16.231:80, len 52 This is my syslog-ng config source s_net{ udp( ); }; destination d_mtksql { sql(type(mysql) host("localhost") username("dbuser") password("dbpasss") database("syslog") table("${R_YEAR}_${R_MONTH}_${R_DAY}") columns( "id int(11) unsigned not null auto_increment primary key", "host varchar(40) not null", "date datetime", "message text not null") values("0", "$FULLHOST", "$R_YEAR-$R_MONTH-$R_DAY $R_HOUR:$R_MIN:$R_SEC", "$MSG") indexes("id")); }; log { source(s_net); destination(d_mtksql); }; i want to separate the message data in different column like- time,username,src mac,src ip,dst ip, nat ip. Any idea how can i achieve that? I am not a pro so if possible kindly write in details how to parse them. Many thanks

2 1