November 2016 - syslog-ng - lists.balabit.hu

Insider 2016-11: Enrichment; Elasticsearch; LISA; FreeBSD; All Things Open;
by Czanik, Péter 11 Nov '16

11 Nov '16

Dear syslog-ng users, This is the 52nd issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Enriching log messages ---------------------- Log messages already contain a wealth of information about what is happening in your systems. Still, logs can be enriched with additional information – like the geolocation belonging to an IP address – which can improve the usability of logs considerably. Read how you can enrich log messages in real-time using syslog-ng at https://www.balabit.com/blog/enriching-log-messages-with-additional-informa… What is new in syslog-ng PE and SSB ----------------------------------- Join us at our next webinar on November 15, where you can learn about new syslog-ng PE and SSB features like content-based alerting, multiple log spaces, key=value parser and Big Data destinations. You can register at http://go.balabit.com/q9VU0A0F5l0E7Q03Z0zY00T Logging to Elasticsearch simplified ----------------------------------- Elasticsearch is gaining the momentum as the ultimate destination for log messages. A great part of it is Kibana, but also that it is a scalable storage of arbitrary name-value pairs. Read, how the use of syslog-ng can greatly simplify how logs reach Elasticsearch: https://www.balabit.com/blog/logging-to-elasticsearch-made-simple-with-sysl… Coming up: LISA --------------- You can learn about syslog-ng at the Balabit booth #145 at LISA (Large Installation System Administration) conference in Boston. Expo days are on December 7 and 8: https://www.usenix.org/conference/lisa16/expo syslog-ng with Java support on FreeBSD -------------------------------------- Java destinations on FreeBSD are not supported by the syslog-ng team and for technical reasons (it downloads JAR files from the Internet) it is also marked broken in FreeBSD ports. Luckily it does not prevent it from working: some people even use it it production. Learn, how to get started at https://www.balabit.com/blog/test-the-latest-freebsd-with-syslog-ng-3-8-1/ All Things Open --------------- Last week I visited All Things Open, one of the largest open source conferences of the US East Coast. I was presenting syslog-ng in the Operations track of the conference, but luckily I had a chance to stay for the full two days of the event. Read my experiences at: TBD Your feedback and news, or tips about the next issue are welcome at documentation(a)balabit.com. To read this newsletter online, visit: https://syslog-ng.org/ Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream https://www.balabit.com/blog/author/peterczanik/ https://twitter.com/PCzanik

1 1

Syslog-ng 3.7.2 install fails on U12.04
by Varugis Kurien 10 Nov '16

10 Nov '16

When installing from the repository here (http://download.opensuse.org/repositories/home:/laszlo_budai:/syslog-ng/xUb…) the install fails to install syslog-ng3.7.2 with the message: rm_conffile voodoo unsupported for package syslog-ng-mod-python! -- This appears to be coming when installing syslog-ng-mod-python due to a failure in a preinstallation script. How can I fix this?

1 1

Possible elementary question
by Varugis Kurien 10 Nov '16

10 Nov '16

I'm having some conniptions doing central logging using syslog-ng. Here is the scenario: 1. I generate logs using syslog-ng on local servers and then send them to a log-destination server. Here is an example of some of the conf files on the generating server: more iptables.log.conf source s_var_log_iptables.log { file("/var/log/iptables.log" program-override(var_log_iptables.log) flags(no-parse)); }; log { source(s_var_log_iptables.log); destination(d_midfin_logger_2); }; more midfin_logger_2.conf template t_sdx_tagged { template("$ISODATE $HOST $PROGRAM $MSG "); template_escape(no); }; destination d_midfin_logger_2 { tcp( "10.4.16.161" port(514) template(t_sdx_tagged)); }; What I am trying to do on the log host is to: 1. Segregate at the top level by host. For each such host: * Split the logs into different directories so that for example, I'll have one directory for the audit log from that host, another for iptables etc - so that under each host I'll have /var/log/iptables.log etc. I've been unable to grok the documentation to figure out quite how to do this and would really appreciate some help. thanks vk ==

3 3

linux-audit-parser not found on stock install of Ubuntu 16.04
by Varugis Kurien 02 Nov '16

02 Nov '16

When using a freshly installed version of U16.04, I found that the linux audit log parser does not seem to be installed. For example: syslog-ng --module-registry|grep audit <-- returns nothing == 1. This started because I experienced a failure when trying to enable the linux audit parser: Error parsing parser expression, parser plugin linux-audit-parser not found in /etc/syslog-ng/conf.d/auditd.log.conf at line 3, column 8: included from /etc/syslog-ng/syslog-ng.conf line 164, column 1 linux-audit-parser(prefix(".auditd.")); 2. The corresponding conf file is below: source s_var_log_audit.log { file("/var/log/audit/audit.log" flags(no-parse)); }; parser p_audit_syslog_parser { linux-audit-parser(prefix(".auditd.")); }; log { source(s_var_log_audit.log); parser(p_audit_syslog_parser); destination(d_midfin_logger_2); }; The officially supported version of syslog-ng in ubuntu is 3.5.1. == syslog-ng 3.5.6 Installer-Version: 3.5.6 Revision: 3.5.6-2.1 [@416d315] (Ubuntu/16.04) Compile-Date: Oct 24 2015 03:49:19 Available-Modules: json-plugin,csvparser,system-source,tfgeoip,afsocket-notls,afamqp,basicfuncs,affile,afsocket-tls,dbparser,afmongodb,cryptofuncs,afsmtp,linux-kmsg-format,afuser,redis,afsocket,afstomp,sysl\ ogformat,confgen,afprog,afsql Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: on Enable-Linux-Caps: on Enable-Pcre: on Do I need to install an unofficial version or is there some elementary mistake that I am making. thanks Varugis

2 1