August 2016 - syslog-ng - lists.balabit.hu

syslog-ng 3.8.1 rpm packages
by Czanik, Péter 15 Sep '16

15 Sep '16

Hi, This week my packaging focus was to make sure, that 3.8.1 will be available as part of the next openSUSE release. My initial unofficial 3.8.1 RPM packages with Java modules, etc. are available but still need some polishing: - move curl (HTTP) module into a separate sub package - update rust modules and on Fedora / EPEL: - build Java modules without Internet access - re-enable missing features, like riemann (which were turned off to make local test builds easier) Fedora/EPEL7: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng38/ EPEL6: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng38epel6/ openSUSE/SLES: https://build.opensuse.org/project/show/home:czanik:syslog-ng38 I hope to have more time working on this by the end of next week. Any feedback, suggestions, patches are welcome Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

4 6

Re: [syslog-ng] Web Visualize logs (Ivan Adji - Krstev)
by Szabó, István 07 Sep '16

07 Sep '16

Hi, For using syslog-ng together with elasticsearch and Kibana this is a pretty good description, also giving you a good understanding of what it enables you to do: https://czanik.blogs.balabit.com/2015/10/how-to-parse-data-with-syslog-ng-s… https://czanik.blogs.balabit.com/2015/12/elasticsearch-and-syslog-ng-fast-a… /Istvan

9 24

Problem with syslog-ng version 2
by Alarie, Maxime 05 Sep '16

05 Sep '16

OS: SLes 11 sp4 syslog-ng: syslog-ng-2.0.9-27.34.39.2 Hi, syslog-ng is configured to read from a application log file and then send it to another file in /var/log (tcpdump is not installed, I cant install it on a blackbox so this is how I test my config) I have the following in my syslog-ng configuration file: source ESRS { file("/opt/esrsve/gateway/xGate.log"); }; destination esrsfile{ file("/var/log/max.log" );}; log { source(ESRS); destination(esrsfile); }; The problem I am seeing is that when syslog writes to the outputfile (esrsfile) it truncatedthe lines. ex: Source File : [ 0, 6, 1, 1007] 08-29-2016 13:56:28.703 IMPORTANT INFO EDDEMC: Data Item::PMStatus Current Value::Offline And the destination file looiks like this: ... ... Aug 29 14:00:02 hostname C Aug 29 14:00:02 hostname u Aug 29 14:00:02 hostname r Aug 29 14:00:02 hostname r Aug 29 14:00:02 hostname e Aug 29 14:00:02 hostname n Aug 29 14:00:02 hostname t Aug 29 14:00:02 hostname Aug 29 14:00:02 hostname V Aug 29 14:00:02 hostname a Aug 29 14:00:02 hostname l Aug 29 14:00:02 hostname u Aug 29 14:00:02 hostname e Aug 29 14:00:02 hostname : Aug 29 14:00:02 hostname : Aug 29 14:00:02 hostname O Aug 29 14:00:02 hostname f Aug 29 14:00:02 hostname f Aug 29 14:00:02 hostname l Aug 29 14:00:02 hostname i Aug 29 14:00:02 hostname n Aug 29 14:00:02 hostname e ... ... What am I doing wrong here? Thanks much!

4 6

Problems with syslog-ng 3.7.3 + mod_confgen
by Jorge Pereira 01 Sep '16

01 Sep '16

Hi guys! Following the sample described in https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-… 1) I have my 'confgen' script that prints the below *file()* entries. (p.s: these files has content.) # /etc/syslog-ng/scripts/confgen-modsec-skeleton.sh file("/opt/nginx/logs/waf/www.cocada.com" program_override("ng_modsec") flags(no-parse)); file("/opt/nginx/logs/waf/www.caipirinha.com" program_override("ng_modsec") flags(no-parse)); # 2) My config set: # cat /etc/syslog-ng/conf.d/nginx_modsec.conf options { threaded(yes); flush_lines(0); use-dns(no); normalize-hostnames(yes); keep-hostname(yes); }; destination d_collector { tcp("192.168.1.248" port(514) keep-alive(on) ); }; log { @module confgen context(source) name(s_nginx_modsec_log) exec("/etc/syslog-ng/scripts/confgen-modsec-skeleton.sh") destination(d_collector); }; # Conclusion: The syslog-ng doesn't call the script at any time. # strace -fff /usr/sbin/syslog-ng -dvte 2>&1 | grep "confgen-modsec" p.s: I have 'confgen' support. # syslog-ng --version | grep confgen Available-Modules: syslogformat,kvformat,afamqp,sdjournal,system-source,afuser,json-plugin,dbparser,affile,afsocket,linux-kmsg-format,afmongodb,mod-python, *confgen* ,csvparser,pseudofile,afsql,afprog,afstomp,cryptofuncs,graphite,basicfuncs # I appreciate any help. Best, Jorge Pereira

3 4

Using patterndb in syslog-ng
by Scot Needy 31 Aug '16

31 Aug '16

Can someone point me in the right direction on how to use update-patterndb for syslog-ng ? Downloaded current git syslog-ng-patterndb to /opt/syslog-ng/etc/patterndb.d/, but not sure how to load and test it. Am I doing something wrong ? bin/pdbtool merge -r --glob \*.pdb -D /opt/syslog-ng/etc/patterndb.d -p /opt/syslog-ng/etc/patterndb.xml [@ROOT] sbin/syslog-ng -f /opt/syslog-ng/etc/syslog-ng.conf [2016-08-31T00:55:54.978717] Non-numeric correlation state ID found, assuming a literal '@' character. To avoid confusion when using a literal '@' after a macro or template function, write '@@' in the template.; Template='${temp.su_username}(a)${temp.xn--su_tty}-e36c blah… blah.. [2016-08-31T00:55:54.978978] Non-numeric correlation state ID found, assuming a literal '@' character. To avoid confusion when using a literal '@' after a macro or template function, write '@@' in the template.; Template='${temp.sudo_username}@unknown’

2 2

Syslog-ng and External variables
by Robert Blayzor 30 Aug '16

30 Aug '16

OS: FreeBSD 10.3 Syslog-ng: 3.7.3 I am trying to reference/use a OS level environment variable in my Syslog-Ng configuration. Everything I’ve read and tried, does not appear to work. I have confirmed simple environment variables like HOSTNAME and PATH are there, but when trying to reference them using backticks, ie: `HOSTNAME` does not appear to work. According to the documentation, environment variables are imported. ie: “Tip The environmental variables of the host are automatically imported and can be used as global variables.” But this does not appear to be the case. When I try referencing a known environment variable I get: Error parsing rewrite expression, syntax error, unexpected KW_VALUE, expecting LL_IDENTIFIER or LL_STRING in /usr/local/etc/syslog-ng.conf at line 56, column 24: rewrite r_host { set(`HOSTNAME`, value("HOST")); }; Or is there another easier way to override the HOST in all messages to be that of the actual servers hostname. (and not things like localhost, 127.0.0.1, etc) TIA -- Robert inoc.net!rblayzor XMPP: rblayzor.AT.inoc.net PGP Key: 78BEDCE1 @ pgp.mit.edu

4 8

Re: [syslog-ng] Syslog-ng and External variables
by Scheidler, Balázs 29 Aug '16

29 Aug '16

Only at config parse time. Regarding the back tick case, are you sure HOSTNAME is set for the syslog-ng process itself? Or is it only set for your interactive shell. On Linux you can check this in proc, /proc/<pid>/env, but I guess similar exists on freebsd. On Aug 29, 2016 6:17 PM, "Robert Blayzor" <rblayzor.bulk(a)inoc.net> wrote: On Aug 29, 2016, at 11:38 AM, Evan Rempel <erempel(a)uvic.ca> wrote: > > @module confgen context(rewrite) name(r_myself) exec("/bin/hostname") I guess the only remaining question on this is a performance one. Would the exec be done for each and every log message or is it done only once? -- Robert inoc.net!rblayzor XMPP: rblayzor.AT.inoc.net PGP Key: 78BEDCE1 @ pgp.mit.edu ____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

1 0

Which version of syslog-ng support TLS1.2?
by Mark Mulligan 24 Aug '16

24 Aug '16

We are using using syslog-ng 3.2.5 and I and trying to figure out how to configure it to use TLS1.2 rather than the TLS1.1. It is working for TLS1.1 but not for connections using TLS1.2, this is a snippet from our config file. source s_waf { tcp(ip(xx.xx.xx.xx) port(xxxx) tls( key_file("/opt/syslog-ng/etc/certs/mux/privateKey.key") cert_file("/opt/syslog-ng/etc/certs/mux/splcollect-01.pem")) ); }; What versions of syslog-ng supports TLS1.2? The version we are using was the latest RH rpm I could find syslog-ng-3.2.5-4.el6.x86_64.rpm I would rather not compile it as it looks like most of the dependencies are also not rpms. Mark Mulligan Assoc Systems Engineer LL Bean Inc. Ext 25077 Cell 249-3823

3 2

Re: [syslog-ng] Elasticsearch destination and date macros
by Fabien Wernli 22 Aug '16

22 Aug '16

On Mon, Aug 22, 2016 at 06:00:29PM +0000, Adam Carter wrote: > I followed a couple of guides to set it up originally, now clearly from old versions. The (latest) online documentation lead me down the right path - thanks for the pointer great to hear that the documentation still does its job so well :-)

1 0

Elasticsearch destination and date macros
by Adam Carter 22 Aug '16

22 Aug '16

Hi All, I am having a problem getting the date macros (Year, Month, Day) to work in syslog-ng OSE when using the elasticsearch plugin. They work for file based outputs so I know the syntax is correct and being parsed correctly. I have now upgraded from syslog-ng 3.6 to syslog-ng 3.8.1, running on Ubuntu 16.04 but still the same symptoms. It is like the ${YEAR} is not being passed as a variable value For example I tried using the index name "syslog-ng-${YEAR}", starting syslog-ng with -Fevd: [2016-08-22T17:26:35.440602] Sending destination program a TERM signal; cmdline='/usr/share/syslog-ng/include/scl/elasticsearch/es-bridge localhost 9200 syslog-ng-${YEAR} syslog-ng', child_pid='12134' The index created in elasticsearch: yellow open syslog-ng- 5 1 239611 0 69.7mb 69.7mb Very simple syslog-ng.conf: destination d_elastic { elasticsearch(index("syslog-ng-${YEAR}") type("syslog-ng") ); }; log { source(s_netsyslog); destination(d_elastic); }; Is this a bug or have I missed something? Thanks

2 1