- syslog-ng - lists.balabit.hu

Is it possible to append a new jSON field?
by Jorge Pereira 05 Sep '16

05 Sep '16

Hi team, Currently, I receive a jSON log from X, but I would like to append a new field. is it possible? e.g: I am trying to do something like: log { source { file("/var/log/app.jsonlog" program_override("ng_app") follow_freq(1) flags(no-parse) template("$(format-json --pair newfield=\"value\")"); ); }; destination(d_remote_collector); };

3 3

Problem with syslog-ng version 2
by Alarie, Maxime 05 Sep '16

05 Sep '16

OS: SLes 11 sp4 syslog-ng: syslog-ng-2.0.9-27.34.39.2 Hi, syslog-ng is configured to read from a application log file and then send it to another file in /var/log (tcpdump is not installed, I cant install it on a blackbox so this is how I test my config) I have the following in my syslog-ng configuration file: source ESRS { file("/opt/esrsve/gateway/xGate.log"); }; destination esrsfile{ file("/var/log/max.log" );}; log { source(ESRS); destination(esrsfile); }; The problem I am seeing is that when syslog writes to the outputfile (esrsfile) it truncatedthe lines. ex: Source File : [ 0, 6, 1, 1007] 08-29-2016 13:56:28.703 IMPORTANT INFO EDDEMC: Data Item::PMStatus Current Value::Offline And the destination file looiks like this: ... ... Aug 29 14:00:02 hostname C Aug 29 14:00:02 hostname u Aug 29 14:00:02 hostname r Aug 29 14:00:02 hostname r Aug 29 14:00:02 hostname e Aug 29 14:00:02 hostname n Aug 29 14:00:02 hostname t Aug 29 14:00:02 hostname Aug 29 14:00:02 hostname V Aug 29 14:00:02 hostname a Aug 29 14:00:02 hostname l Aug 29 14:00:02 hostname u Aug 29 14:00:02 hostname e Aug 29 14:00:02 hostname : Aug 29 14:00:02 hostname : Aug 29 14:00:02 hostname O Aug 29 14:00:02 hostname f Aug 29 14:00:02 hostname f Aug 29 14:00:02 hostname l Aug 29 14:00:02 hostname i Aug 29 14:00:02 hostname n Aug 29 14:00:02 hostname e ... ... What am I doing wrong here? Thanks much!

4 6

Is it possible to append a new jSON field?
by Jorge Pereira 04 Sep '16

04 Sep '16

Hi team, Currently, I receive a jSON log from X, but I would like to append a new field. is it possible? -- Jorge Pereira

1 0

[GSoC 2016] Kafka source in Java
by Vithulan MV 01 Sep '16

01 Sep '16

Hi all, I'd like to take this moment to Thank everyone for all the support and guidance provided to me throughout this period. Specially for my mentor Viktor Juhász and of course to the syslog-ng community! This was a wonderful journey with lot of experiences. Blogpost about Kafka source : https://vithulanmv.wordpress.com/2016/08/21/syslog-ng-kafka-source-in-depth… I am hoping to continue my contribution to syslog-ng project :) Thank you. Best regards, Vithulan. -- Vithulan MV. Undergraduate, Department of Computer Science & Engineering, University of Moratuwa, Sri lanka.

1 0

Problems with syslog-ng 3.7.3 + mod_confgen
by Jorge Pereira 01 Sep '16

01 Sep '16

Hi guys! Following the sample described in https://www.balabit.com/documents/syslog-ng-ose-latest-guides/en/syslog-ng-… 1) I have my 'confgen' script that prints the below *file()* entries. (p.s: these files has content.) # /etc/syslog-ng/scripts/confgen-modsec-skeleton.sh file("/opt/nginx/logs/waf/www.cocada.com" program_override("ng_modsec") flags(no-parse)); file("/opt/nginx/logs/waf/www.caipirinha.com" program_override("ng_modsec") flags(no-parse)); # 2) My config set: # cat /etc/syslog-ng/conf.d/nginx_modsec.conf options { threaded(yes); flush_lines(0); use-dns(no); normalize-hostnames(yes); keep-hostname(yes); }; destination d_collector { tcp("192.168.1.248" port(514) keep-alive(on) ); }; log { @module confgen context(source) name(s_nginx_modsec_log) exec("/etc/syslog-ng/scripts/confgen-modsec-skeleton.sh") destination(d_collector); }; # Conclusion: The syslog-ng doesn't call the script at any time. # strace -fff /usr/sbin/syslog-ng -dvte 2>&1 | grep "confgen-modsec" p.s: I have 'confgen' support. # syslog-ng --version | grep confgen Available-Modules: syslogformat,kvformat,afamqp,sdjournal,system-source,afuser,json-plugin,dbparser,affile,afsocket,linux-kmsg-format,afmongodb,mod-python, *confgen* ,csvparser,pseudofile,afsql,afprog,afstomp,cryptofuncs,graphite,basicfuncs # I appreciate any help. Best, Jorge Pereira

3 4

Using patterndb in syslog-ng
by Scot Needy 31 Aug '16

31 Aug '16

Can someone point me in the right direction on how to use update-patterndb for syslog-ng ? Downloaded current git syslog-ng-patterndb to /opt/syslog-ng/etc/patterndb.d/, but not sure how to load and test it. Am I doing something wrong ? bin/pdbtool merge -r --glob \*.pdb -D /opt/syslog-ng/etc/patterndb.d -p /opt/syslog-ng/etc/patterndb.xml [@ROOT] sbin/syslog-ng -f /opt/syslog-ng/etc/syslog-ng.conf [2016-08-31T00:55:54.978717] Non-numeric correlation state ID found, assuming a literal '@' character. To avoid confusion when using a literal '@' after a macro or template function, write '@@' in the template.; Template='${temp.su_username}(a)${temp.xn--su_tty}-e36c blah… blah.. [2016-08-31T00:55:54.978978] Non-numeric correlation state ID found, assuming a literal '@' character. To avoid confusion when using a literal '@' after a macro or template function, write '@@' in the template.; Template='${temp.sudo_username}@unknown’

2 2

Syslog-ng and External variables
by Robert Blayzor 30 Aug '16

30 Aug '16

OS: FreeBSD 10.3 Syslog-ng: 3.7.3 I am trying to reference/use a OS level environment variable in my Syslog-Ng configuration. Everything I’ve read and tried, does not appear to work. I have confirmed simple environment variables like HOSTNAME and PATH are there, but when trying to reference them using backticks, ie: `HOSTNAME` does not appear to work. According to the documentation, environment variables are imported. ie: “Tip The environmental variables of the host are automatically imported and can be used as global variables.” But this does not appear to be the case. When I try referencing a known environment variable I get: Error parsing rewrite expression, syntax error, unexpected KW_VALUE, expecting LL_IDENTIFIER or LL_STRING in /usr/local/etc/syslog-ng.conf at line 56, column 24: rewrite r_host { set(`HOSTNAME`, value("HOST")); }; Or is there another easier way to override the HOST in all messages to be that of the actual servers hostname. (and not things like localhost, 127.0.0.1, etc) TIA -- Robert inoc.net!rblayzor XMPP: rblayzor.AT.inoc.net PGP Key: 78BEDCE1 @ pgp.mit.edu

4 8

Re: [syslog-ng] Syslog-ng and External variables
by Scheidler, Balázs 29 Aug '16

29 Aug '16

Only at config parse time. Regarding the back tick case, are you sure HOSTNAME is set for the syslog-ng process itself? Or is it only set for your interactive shell. On Linux you can check this in proc, /proc/<pid>/env, but I guess similar exists on freebsd. On Aug 29, 2016 6:17 PM, "Robert Blayzor" <rblayzor.bulk(a)inoc.net> wrote: On Aug 29, 2016, at 11:38 AM, Evan Rempel <erempel(a)uvic.ca> wrote: > > @module confgen context(rewrite) name(r_myself) exec("/bin/hostname") I guess the only remaining question on this is a performance one. Would the exec be done for each and every log message or is it done only once? -- Robert inoc.net!rblayzor XMPP: rblayzor.AT.inoc.net PGP Key: 78BEDCE1 @ pgp.mit.edu ____________________________________________________________ __________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/? product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

1 0

Which version of syslog-ng support TLS1.2?
by Mark Mulligan 24 Aug '16

24 Aug '16

We are using using syslog-ng 3.2.5 and I and trying to figure out how to configure it to use TLS1.2 rather than the TLS1.1. It is working for TLS1.1 but not for connections using TLS1.2, this is a snippet from our config file. source s_waf { tcp(ip(xx.xx.xx.xx) port(xxxx) tls( key_file("/opt/syslog-ng/etc/certs/mux/privateKey.key") cert_file("/opt/syslog-ng/etc/certs/mux/splcollect-01.pem")) ); }; What versions of syslog-ng supports TLS1.2? The version we are using was the latest RH rpm I could find syslog-ng-3.2.5-4.el6.x86_64.rpm I would rather not compile it as it looks like most of the dependencies are also not rpms. Mark Mulligan Assoc Systems Engineer LL Bean Inc. Ext 25077 Cell 249-3823

3 2

Re: [syslog-ng] Elasticsearch destination and date macros
by Fabien Wernli 22 Aug '16

22 Aug '16

On Mon, Aug 22, 2016 at 06:00:29PM +0000, Adam Carter wrote: > I followed a couple of guides to set it up originally, now clearly from old versions. The (latest) online documentation lead me down the right path - thanks for the pointer great to hear that the documentation still does its job so well :-)

1 0