- syslog-ng - lists.balabit.hu

syslog 3.9 modifying ${MSG}?
by Nik Ambrosch 27 Apr '17

27 Apr '17

After moving from syslog-ng 3.5 to 3.9 i noticed that the contents of $PROGRAM and $MSG are being logged differently than before. Here is how they used to be logged: # program: 53 # message: Apr 19 09:35:35.713 GMT: %ILPOWER-5-POWER_GRANTED: Interface xxx: Power granted | That is the full message (as seen on the device) which is optimal behavior. Below is the behavior i’m seeing with syslog-ng 3.9 with similar configuration: # program: GMT # message: %ILPOWER-5-POWER_GRANTED: Interface xxx: Power granted The value of program and message are altered but everything else is the same. I’ve been investigating the date parser and the flags(no-parse) options but haven’t had any luck getting a properly formatted message yet. If anyone has any ideas on how to get the old behavior back it would be greatly appreciated.

3 5

Convert logstash filters to syslog-ng
by C. L. Martinez 26 Apr '17

26 Apr '17

Hi all, I would like to drop Logstash collector from our ELK infrastructure and use syslog-ng instead. This ELK infrastructure collects, report and show dashboards about security devices: firewalls, anti-spam devices, etc. Most of these logs arrives from rsyslog collectors (deployed in several linux and BSD machines). I have seen in Balabit's blog page how this could be done: https://www.balabit.com/blog/how-to-parse-data-with-syslog-ng-store-in-elas… and https://www.balabit.com/blog/collecting-and-parsing-suricata-logs-using-sys…. The most important point here is to test all configured logstash filters inside syslog-ng: GeoIP patterns, some substitution params, etc. Any tips or tricks to accomplish this type of change? Many thanks. -- Greetings, C. L. Martinez

4 3

Getting log statistics file every 10 minutes.
by Jordan Ladora 25 Apr '17

25 Apr '17

Hello, After running normally for 6 or 7 days, starting at midnight, on my syslog-ng server running freebsd 11-Rel and syslog-ng 3.7.3_7 (installed with pkg), I started getting a Log statistics record every 10 minutes, which is excessive. This is only happening with a local log file designated in the .conf file like so- source s_local { system(); internal(); }; destination d_local { file("/var/log/sng_local"); }; log { source(s_local); destination(d_local); }; This is one of the entries I'm getting every 10min in the /var/log/sng_local file- Apr 25 12:00:02 logbox syslog-ng[370]: Log statistics; processed='source(s_network)=2602643', processed='destination(security)=18', processed='center(received)=2912379', processed='destination(ppp)=0', processed='src.internal(s_local#3)=894', stamp='src.internal(s_local#3)=1493139002', processed='center(queued)=3235133', processed='destination(console)=15635', processed='src.none()=0', stamp='src.none()=0', processed='destination(audit)=18', processed='destination(messages)=2595805', processed='destination(xferlog)=0', processed='global(payload_reallocs)=1703', processed='global(sdata_updates)=27054', processed='destination(allusers)=0', processed='destination(authlog)=391', processed='destination(maillog)=299021', processed='destination(consolelog)=0', processed='destination(lpd-errs)=0', processed='destination(cron)=14282', processed='destination(slip)=0', processed='global(msg_clones)=0', processed='source(s_local)=309736', processed='destination(d_local)=309736', processed='destination(debuglog)=227', processed='global(internal_queue_length)=0' What did I do to deserve this every 10 minutes?

2 2

Re: [syslog-ng] Unable to Filter Based On Facility into Different Files
by james.r.hendrick 21 Apr '17

21 Apr '17

flags (final) stops the professing in the first statement Sent from my Verizon, Samsung Galaxy smartphone -------- Original message --------From: wiskbroom(a)hotmail.com Date: 4/21/17 9:37 AM (GMT-05:00) To: syslog-ng(a)lists.balabit.hu Subject: [syslog-ng] Unable to Filter Based On Facility into Different Files Greetings! I am trying to rewrite syslog-ng.conf to create files based on facilities; one way for non-auth messages, another for all authentication messages (ssh, su, sudo, and console logins). I believe I have two issues with my statements below: 1. My ${HOST}- might be incorrect. 2. Am I able to write two filters for a single source? My single source in this case are Linux boxes, all sending their syslog traffic to my syslog-NG server with *.*. My statements below, comments and criticism very welcome. filter f_linux_secure { facility(authpriv) and level(info..emerg); }; filter f_linux_messages { level(info..emerg); }; destination d_linux_secure { file("/data/Linux/${HOST}-secure.log" owner("root") group("systems") perm(0640) dir_perm(0750) create_dirs(yes)); destination d_linux_messages { file("/data/Linux/${HOST}-messages.log" owner("root") group("systems") perm(0640) dir_perm(0750) create_dirs(yes)); log { source(s_remote); filter(f_linux_secure); destination(d_linux_secure); flags(final); }; log { source(s_remote); filter(f_linux_messages); destination(d_linux_messages); flags(final); }; Regards, Vadim Anatoly Pushkin

3 3

Unable to Filter Based On Facility into Different Files
by wiskbroom＠hotmail.com 21 Apr '17

21 Apr '17

Greetings! I am trying to rewrite syslog-ng.conf to create files based on facilities; one way for non-auth messages, another for all authentication messages (ssh, su, sudo, and console logins). I believe I have two issues with my statements below: 1. My ${HOST}- might be incorrect. 2. Am I able to write two filters for a single source? My single source in this case are Linux boxes, all sending their syslog traffic to my syslog-NG server with *.*. My statements below, comments and criticism very welcome. filter f_linux_secure { facility(authpriv) and level(info..emerg); }; filter f_linux_messages { level(info..emerg); }; destination d_linux_secure { file("/data/Linux/${HOST}-secure.log" owner("root") group("systems") perm(0640) dir_perm(0750) create_dirs(yes)); destination d_linux_messages { file("/data/Linux/${HOST}-messages.log" owner("root") group("systems") perm(0640) dir_perm(0750) create_dirs(yes)); log { source(s_remote); filter(f_linux_secure); destination(d_linux_secure); flags(final); }; log { source(s_remote); filter(f_linux_messages); destination(d_linux_messages); flags(final); }; Regards, Vadim Anatoly Pushkin

1 0

RSS memory grows when disk is full
by Arnob Ghosh 20 Apr '17

20 Apr '17

Hi, We are currently using syslog-ng version 3.3.6 . We are observing RSS memory growing issue for syslog-ng when destination disk is full. Is that an expected behavior , as the incoming message gets queued up in system RAM and thus RSS also will grow for syslog-ng process. Please correct me if I am wrong in understanding the same. Thanks, Arnob

1 0

down network during reload leads to blocking syslog() calls
by Nathan Parrish 18 Apr '17

18 Apr '17

Hi there, we are using syslog-ng 3.6.4 on Linux. we had an incident where the network port used for remote logging was inadvertently disabled for a couple hours, and during this time a critical process which logged via syslog() calls experienced threads hanging for seconds to minutes at a time. after some investigation and reproduction efforts, it looks like the problem is a combination of: - remote logging to server specified by hostname (vs. IP) - loss of management interface (e.g. that used for syslog traffic and DNS resolution) - log rotate triggering syslog-ng reload when I reproduce the problem (seems to take some considerable amount of logging load and a minute or so), I can see that writes to /dev/log hang: root@escort-ct0:/etc# strace -ttT logger test ^C 19:38:46.064830 execve("/usr/bin/logger", ["logger", "test"], [/* 26 vars */]) = 0 <0.000135> ... 19:38:46.069757 socket(PF_LOCAL, SOCK_DGRAM|SOCK_CLOEXEC, 0) = 1 <0.000015> 19:38:46.069806 connect(1, {sa_family=AF_LOCAL, sun_path="/dev/log"}, 110) = 0 <0.000010> 19:38:46.069850 sendto(1, "<13>Mar 20 19:38:46 root: test", 30, MSG_NOSIGNAL, NULL, 0) = 30 <11.627190> 19:38:57.697110 close(1)= 0 <0.000490> meanwhile I see the main thread timing out name resolution: 19:48:18.214182 stat("/etc/resolv.conf", {st_mode=S_IFREG|0644, st_size=148, ...}) = 0 <0.000008> 19:48:18.214234 socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 31 <0.000012> 19:48:18.214279 connect(31, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("10.15.83.11")}, 16) = 0 <0.000024> 19:48:18.214332 poll([{fd=31, events=POLLOUT}], 1, 0) = 1 ([{fd=31, revents=POLLOUT}]) <0.000006> 19:48:18.214370 sendto(31, "\343\235\1\0\0\1\0\0\0\0\0\0\rescdev-syslog\3dev\vpurestorage\3com\0\0\1\0\1", 51, MSG_NOSIGNAL, NULL, 0) = 51 <0.000047> 19:48:18.214444 poll([{fd=31, events=POLLIN}], 1, 5000) = 0 (Timeout) <5.005040> this repeats with the other DNS servers configured. I cannot reproduce the issue if I configure an IP instead of hostname for the syslog server. we are using UDP and no flow-control configuration, with the expectation that syslog() will never block. and, indeed, until the reload, it works as expected. however, after reload I guess we re-establish the socket for the remote connection, requiring us to resolve the hostname; I don’t pretend to understand how this ultimately backs up processing of /dev/log (note that internal() and kernel messages are coming through just fine during this time). I guess my question is whether this is a known/expected issue, and/or if there’s a resolution other than specifying remote syslog servers by IP or hardcoding the name resolution in /etc/hosts and pointing to that with dns-cache-hosts(). basically, I’d like syslog-ng to simply give up if it can’t resolve remote syslog server hostnames, rather than allow this to interfere with servicing of /dev/log, with ramifications to callers. thanks in advance, nathan

2 1

Insider 2017-04: Docker; Splunk HEC; Suricata; videos; upcoming events;
by Czanik, Péter 13 Apr '17

13 Apr '17

Dear syslog-ng users, This is the 57th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Central log server in Docker ---------------------------- Containerization, and Docker in particular, changed the way we distribute and run applications. Your central log server can also run in a Docker container. If you wish to deploy your log server running syslog-ng in a Docker container, it is available as a ready-to-use image from the Docker Hub, already passing 500K pulls. Read how at https://www.balabit.com/blog/central-log-server-docker/ Feeding the Splunk HTTP event collector --------------------------------------- Using the HTTP destination of syslog-ng, you can feed the Splunk HTTP event collector (HEC) with log messages. This can simplify your logging architecture, because there is no need to store data in files and to use a Splunk forwarder. Read more about how to configure it on the Splunk website at https://www.splunk.com/blog/2017/03/30/syslog-ng-and-hec-scalable-aggregate… syslog-ng PE 7.0.2 available ---------------------------- The latest syslog-ng Premium Edition release adds two interesting new features: you can write message parsers and template functions in Python and the monitoring() source allows you to granularly select which statistics of syslog-ng PE you want to monitor. https://www.balabit.com/blog/monitor-your-syslog-ng-architecture-easier/ Collecting and parsing Suricata logs ------------------------------------ You can use syslog-ng to collect and parse the JSON-based log messages of Suricata. Learn how you can send these logs to Loggly or Elasticsearch for further analysis or configure simple alerting within syslog-ng: https://www.balabit.com/blog/collecting-and-parsing-suricata-logs-using-sys… Tutorial videos --------------- Patrick Bailey (https://twitter.com/whiteboardcoder) created a couple of tutorial videos about syslog-ng. These cover installation and initial configuration on Ubuntu, opening a network port for collecting log messages and sending JSON-based log messages: - https://www.youtube.com/watch?v=DrfBU9nBeoE - https://www.youtube.com/watch?v=YAYfBteY0kg - https://www.youtube.com/watch?v=Uejb0agO2NU UPCOMING EVENTS You can learn about syslog-ng at a growing number of events: HEPIX Spring Workshop: https://indico.cern.ch/event/595396/ Big Data Universe: https://bdu.hu/ openSUSE conference: https://events.opensuse.org/conference/oSC17 NEW RELEASES syslog-ng PE 7.0.2: https://www.balabit.com/documents/syslog-ng-pe-7.0-guides/en/syslog-ng-pe-g… Your feedback and news, or tips about the next issue are welcome at documentation(a)balabit.com. To read this newsletter online, visit: https://syslog-ng.org/ Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

1 0

syslog-ng buffer and reload
by Fabien Wernli 13 Apr '17

13 Apr '17

Hi, I'm using the disk queue to buffer the writes to elasticsearch. I notices the following behaviour: When reloading the configuration (`syslog-ng-ctl reload`), syslog-ng stops processing incoming messages (they appear as `dropped` in stats) and starts emptying the queue. It only starts accepting new messages when the queue is completely empty. I understand this is probably an expected behaviour, but in the following scenario (I just experienced) it poses a problem: 1. some application goes bananas logging at zillions of events per second 2. syslog-ng queue starts filling up 3. crazy app identified: I modify syslog-ng.conf in order to filter out the app 4. syslog-ng-ctl reload 5. syslog-ng starts dropping all new messages and emptying the queue 6. I have to wait for the queue to be empty (which can take a long time) Wouldn't it be saner to continue accepting messages when intercepting the HUP?

4 6

is it exist some way to extract the filename listened by file() ?
by Jorge Pereira 08 Apr '17

08 Apr '17

Hi Team, Well, I am working on a POC using the syslog-ng 3.7.1, basically, I have many of log files that the filename is /path/<file> and I need to append the file name into the syslog payload. My current approach is. 1. I have the below destination() receiving the file name as a parameter. <snip> block destination d_collector_with_fn(__filename("")) { tcp("192.168.2.44" port(514) keep-alive(on) template("$DATE $HOST $MSGHDR $(format-json --scope selected_macros \ --exclude TAGS \ --exclude DATE \ --exclude PRIORITY \ --exclude FACILITY \ --exclude SOURCEIP \ --exclude PROGRAM \ --pair SYSLOG_WEBAPP_DOMAIN='`__filename`' \ --pair SOURCE=${SOURCE} )\n") template-escape(no) ); }; </snip> 2. My simple script called by confgen create some dynamic "log {}" statements listening to the files and appending the filename as a parameter to the d_collector_with_fn() <snip> log { source { file("/path/thisisafile001.net" program_override("mytag") follow_freq(1) flags(no-parse) ); }; destination { d_collector_with_fn(__filename("thisisafile001.net")); }; }; log { source { file("caipirinha4ever.net" program_override("mytag") follow_freq(1) flags(no-parse) ); }; destination { d_collector_with_fn(__filename("caipirinha4ever.net")); }; }; ......................... </snip> But, I have more than 5k files and my current approach creating multiples log { } statement resulting in one connection to the collector by each file!!! in this case, I have 5k connections... this is terrible, someone has some other suggestion? exist some way to catch the filename by some internal ${variable} and pass for a single destination()? -- Jorge Pereira

3 7