February 2016 - syslog-ng - lists.balabit.hu

powered by syslog-ng
by Czanik, Péter 19 Jul '16

19 Jul '16

Hi, We have now a "Powered by syslog-ng" page on our website. It's available at https://syslog-ng.org/powered-by-syslog-ng/ If your project / company / product / etc. uses syslog-ng, we are happy to include you on the list too, just e-mail me directly! Some of the more interesting names from the list are Facebook, Kindle, FreeNAS and of course all the major Linux distributions. Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

2 1

syslog-ng 3.7.2
by Budai, László 22 Mar '16

22 Mar '16

3.7.2 This is the first maintenance release for the 3.7.x series. Changes compared to 3.7.1: Improvements - Added mbox() source. This source can be used to fetch emails from local mbox files: source { mbox("/var/spool/mail/root"); }; This will fetch root emails and parse them into a multiline $MSG. Original implementation by Fabien Wernli, I only converted it into an SCL. - It is possible to append dynamically options into SCL blocks from now. - concurrent_request option added to ElasticSearch with default value 1. - In elasticsearch destinaton, message_template() argument renamed to template(). - SCL added to every Java module (ElasticSearch, Kafka, HDFS). - Linux Audit Parser added for parsing key-value pairs produced by the Linux Audit subsystem. - HTTP destination is now able to receive HTTP method as an option. All the supported methods are available (POST, PUT, HEAD, OPTIONS, DELETE, TRACE, GET). Fixes - In some circumstances syslog-ng mod-journal re-read every already processed messages. - When syslog-ng got a reload and the reload process done within 1 second then mafter the reload, syslog-ng stop generating mark-messages. - When initialization of a network destination in syslog-ng failed (eg. due to DNS resolution failure) we didn't create a queue which caused message loss. - syslog-ng segfaulted on TLS errors when wrong certs was provided (eg.: CA cert with the cert-file directive instead of the server cert). - Fixed a continuous spinning case in the file driver, when the destination file is a device (e.g. /dev/stdout). - A memory leak in around template functions in grammar fixed. - Fixed Python3 support. - Fixed Python GIL issue in python destination. - From now, instead of skipping doc/ alltogether when ENABLE_MANPAGES is not set, only skip the actual man pages, but handle the rest properly. - Allow overriding the python setup.py options. When installing the python modules, allow overriding the options. This is useful for distributions that want to pass extra options. For example, on Debian, we want --install-layout="deb" instead of the --prefix and --root options. With this change, the previous behaviour remains the default, but one can supply PYSETUP_OPTIONS on the make command-line to override it. - The systemd service file read /etc/default/syslog-ng and /etc/sysconfig/syslog-ng, but didn't do anything with their contents. $SYSLOGNG_OPTS added to ExecStart, so that the EnvironmentFiles have an effect (at least on Debian). - Java support checking fixed (not only jdk is required but also gradle). - Memory leak around ping() in Redis fixed. - A crash in pdbtool fixed around r_parser_email(). - Removed cygwin fdlimit statement. Make the default for RLIMIT_NOFILE equal to the current system limits. --fd-limit can still override this, but the default will be configured based on existing system limits. - Fixed BSD year inference. Fixed logic and made clearer the inference of year from bsd-style rfc3164 syslog-messages, which do not include a year. - Handle correctly the epoch 0 timestamp. (Previously, syslog-ng cached the zero timestamp and treated 1970 as it was 1900.) Credits syslog-ng is developed as a community project, and as such it relies on volunteers, to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. We would like to thank the following people for their contribution: Adam Arsenault, Adam Istvan Mozes, Andras Mitzki, Avleen Vig, Balazs Scheidler, Fabien Wernli, Gergely Czuczy, Gergely Nagy, Gergo Nagy, Laszlo Budai, Peter Czanik, Robert Fekete, Saurabh Shukla, Tamas Nagy, Tibor Benke, Viktor Juhasz, Vincent Bernat, Wang Long, Zdenek Styblik, Zoltan Pallagi. — View it on GitHub <https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.7.2>.

4 3

Error loading Python module for log destination
by Wil Koch 29 Feb '16

29 Feb '16

Hi All, I am following the tutorial below in an attempt to get Python support for destinations, https://syslog-ng.gitbooks.io/getting-started/content/chapters/chapter_5/se… I have built syslog-ng-3.7.2 from source and have it up and running on Ubuntu 14.04. I initially had a simple log for iptables logging to a file to confirm functionality. After I changed my conf to include a Python destination, I am receiving the following error when starting syslog-ng at the command line, [2016-02-26T12:15:48.874748] Error loading Python module; module='betterpythonexample', exception='exceptions.ImportError: No module named betterpythonexample' [2016-02-26T12:15:48.874775] Error looking Python driver class; driver='python_to_file#0', class='betterpythonexample.TextDestination', exception='None' [2016-02-26T12:15:48.874782] Error initializing message pipeline; I have PYTHONPATH set to the directory where the script is, and I have confirmed the variable is working by importing the module in the python interpreter. Any suggestions for debugging would be much appreciated, I'm out of ideas. Thank you, Wil

2 2

GSoC 2016
by Laszlo Budai 29 Feb '16

29 Feb '16

Hi, the syslog-ng project has been accepted as a mentoring organization :) During the next few days we will contact you, as we want to create a special GSoC syslog-ng release ( https://syslog-ng.org/gsoc-2016-applicant-form-syslog-ng-project ). regards, Laszlo Budai Sent from my HTC

1 0

simple subscription based interface?
by Árpád Magosányi 29 Feb '16

29 Feb '16

Hi, The use case is a crystal box: everything (but keys) are public in the system, including logs. So I want to configure syslog-ng such that anyone can connect to it, and receives the logs. The simpler the interface, the better. I also would like to have some limit on the number of concurrent clients. Any way to achieve this with syslog-ng without using an additional stomp or amqp server? My syslog-ng version is 3.5.6-2. If there is a need for a message broker, could anyone suggest a free or dirt cheap one in the cloud?

3 3

Error while configuring destination for network
by Girish Kumar 29 Feb '16

29 Feb '16

Hi All, In the conf file I have added destination as network. But I am getting following error while spawning the syslog-ng . Could you please help me on this [root@localhost vsabarin]# /sbin/syslog-ng Error parsing afsocket, syntax error, unexpected KW_IP, expecting LL_IDENTIFIER or LL_STRING in /etc/syslog-ng/syslog-ng.conf at line 12, column 32: destination d_network { syslog(ip("135.254.163.151") transport("tcp")); }; My syslog-ng.conf file @version:3.7 @include "scl.conf" source s_local { internal(); }; destination d_network { syslog(ip("135.254.163.151") transport("tcp")); }; destination d_local { file("/var/log/messagesgiri"); }; log { source(s_local); destination(d_local); destination d_network); }; Regards, Girish

2 3

Error while spawning the syslog-ng : "source plugin system not found "
by Girish Kumar 27 Feb '16

27 Feb '16

Hi All, I was successful in compilation and installing syslog-ng While spawning syslog-ng, I am getting source plugin system not found. Following is the error. Please help me on this TOR #-> /sbin/syslog-ng Error parsing source, source plugin system not found in //etc/syslog-ng.conf at line 10, column 2: system(); ^^^^^^ syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng TOR #-> syslog-ng -V syslog-ng 3.7.2 Installer-Version: 3.7.2 Revision: Compile-Date: Feb 24 2016 23:02:43 Available-Modules: json-plugin,kvformat,pseudofile,linux-kmsg-format,csvparser,system-source,afamqp,cryptofuncs,basicfuncs,afuser,afprog,graphite,afstomp,afsocket,syslogformat,confgen,dbparser,sdjournal,affile,afmongodb Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off Regards, Girish

3 7

Error while spawning the syslog-ng : "source plugin system not found "
by Girish Kumar 25 Feb '16

25 Feb '16

Hi All, I was successful in compilation and installing syslog-ng While spawning syslog-ng, I am getting source plugin system not found. Following is the error. Please help me on this TOR #-> /sbin/syslog-ng Error parsing source, source plugin system not found in //etc/syslog-ng.conf at line 10, column 2: system(); ^^^^^^ syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng TOR #-> syslog-ng -V syslog-ng 3.7.2 Installer-Version: 3.7.2 Revision: Compile-Date: Feb 24 2016 23:02:43 Available-Modules: json-plugin,kvformat,pseudofile,linux-kmsg-format,csvparser,system-source,afamqp,cryptofuncs,basicfuncs,afuser,afprog,graphite,afstomp,afsocket,syslogformat,confgen,dbparser,sdjournal,affile,afmongodb Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-IPv6: on Enable-Spoof-Source: off Enable-TCP-Wrapper: off Enable-Linux-Caps: off Regards, Girish

1 0

Error parsing config and compiling template
by amey jain 25 Feb '16

25 Feb '16

Hi, I was running ./syslog-ng and I got these errors. I am unable to solve them. I am copying the output here. Please tell me what to do to correct it. $ sbin/syslog-ng --module-path=/home/amey/install/syslog-ng Error parsing config, Error compiling template (Unknown template function "format-json") in /home/amey/install/syslog-ng/share/syslog-ng/include/scl/cim/template.conf at line 23, column 32: included from /home/amey/install/syslog-ng/etc/scl.conf line 29, column 1 included from /home/amey/install/syslog-ng/etc/syslog-ng.conf line 8, column 1 template-function "format-cim" "$(format-json --pair @timestamp='${R_ISODATE}' --pair @message='${MSG}' --key .cim.* --shift 5 --key _* --key .* --replace-prefix .=_ --key *.*)\n"; ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng Regards

3 2

Re: [syslog-ng] syslog-ng Digest, Vol 130, Issue 35
by Ho, Ming 24 Feb '16

24 Feb '16

Hi Peter, Thanks for the suggestion. I am now able to run "configure" to include Java for complication. But I ran into the following error when I executed "make" command: In file included from modules/java/native/java-destination.h:36, from modules/java/native/java-parser.h:29, from modules/java/native/java-grammar.y:26: ./modules/java/proxies/java-destination-proxy.h:46: error: expected declaration specifiers or '...' before 'JNIEnv' make[2]: *** [modules/java/native/modules_java_libmod_java_la-java-grammar.lo] Error 1 make[1]: *** [all-recursive] Error 1 make: *** [all] Error 2 Appreciate your help. Ming -----Original Message----- From: syslog-ng-bounces(a)lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of syslog-ng-request(a)lists.balabit.hu Sent: Tuesday, February 23, 2016 8:49 AM To: syslog-ng(a)lists.balabit.hu Subject: syslog-ng Digest, Vol 130, Issue 35 Send syslog-ng mailing list submissions to syslog-ng(a)lists.balabit.hu To subscribe or unsubscribe via the World Wide Web, visit https://lists.balabit.hu/mailman/listinfo/syslog-ng or, via email, send a message with subject or body 'help' to syslog-ng-request(a)lists.balabit.hu You can reach the person managing the list at syslog-ng-owner(a)lists.balabit.hu When replying, please edit your Subject line so it is more specific than "Re: Contents of syslog-ng digest..." Today's Topics: 1. Re: syslog-ng Digest, Vol 130, Issue 27 (Czanik) 2. Re: Cross Compiling syslog-ng 3.7.2 on power-pc architecture (Girish Kumar) ---------------------------------------------------------------------- Message: 1 Date: Tue, 23 Feb 2016 13:15:33 +0100 From: Czanik, P?ter <peter.czanik(a)balabit.com> Subject: Re: [syslog-ng] syslog-ng Digest, Vol 130, Issue 27 To: "Ho, Ming" <Mingho(a)visa.com> Cc: "syslog-ng(a)lists.balabit.hu" <syslog-ng(a)lists.balabit.hu> Message-ID: <CANcUavtb3iKWG3Kr9RKOvwLYF5JcO7mPqomfAnuxoUwqsEWTHw(a)mail.gmail.com> Content-Type: text/plain; charset="utf-8" Hi, Gradle needs to be available in the PATH during configure. If it's not there, you should update your PATH variable, like this: export PATH=/usr/share/java/gradle-2.11/bin:$PATH Gradle can download you all the necessary JAR files. If you want to use the local files from another directory, you need to edit modules/java-modules/build.gradle before compilation. Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik On Mon, Feb 22, 2016 at 7:14 PM, Ho, Ming <Mingho(a)visa.com> wrote: > Hi Peter, > > Thanks for the quick response. The admin guide and your blog only > mentioned to install gradle, which I did, but no mention of how to use > it to build Java modules. I am not a software developer and have not > heard/use gradle before. It will be very kind and helpful of you if > you can provide me the steps how to do so. And any additional steps I > need to do for compilation after the Java modules created. > > I also try your syslog-ng-java-hack-1.0-1.el6.x86_64.rpm. I copy all > the jars files to /usr/local/lib/syslog-ng/java-modules and then run > "configure" but the Java module is still not included. What is the > correct steps if I use the provided jar files? > > Thanks. > Ming > > -----Original Message----- > From: syslog-ng-bounces(a)lists.balabit.hu [mailto: > syslog-ng-bounces(a)lists.balabit.hu] On Behalf Of > syslog-ng-request(a)lists.balabit.hu > Sent: Saturday, February 20, 2016 3:00 AM > To: syslog-ng(a)lists.balabit.hu > Subject: syslog-ng Digest, Vol 130, Issue 27 > > Send syslog-ng mailing list submissions to > syslog-ng(a)lists.balabit.hu > > To subscribe or unsubscribe via the World Wide Web, visit > https://lists.balabit.hu/mailman/listinfo/syslog-ng > or, via email, send a message with subject or body 'help' to > syslog-ng-request(a)lists.balabit.hu > > You can reach the person managing the list at > syslog-ng-owner(a)lists.balabit.hu > > When replying, please edit your Subject line so it is more specific > than > "Re: Contents of syslog-ng digest..." > > > Today's Topics: > > 1. Re: Not able to compile syslog-ng OSE with Java support (Czanik) > 2. ECONNREFUSED on /dev/log (?rp?d Magos?nyi) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Sat, 20 Feb 2016 07:47:23 +0100 > From: Czanik, P?ter <peter.czanik(a)balabit.com> > Subject: Re: [syslog-ng] Not able to compile syslog-ng OSE with Java > support > To: "Syslog-ng users' and developers' mailing list" > <syslog-ng(a)lists.balabit.hu> > Message-ID: > < > CANcUavtxRVrSQwcaAB0WV27uO+gsR1T7kT8rQqq63Rev+ns+Ag(a)mail.gmail.com> > Content-Type: text/plain; charset="utf-8" > > You will also need gradle to build Java modules: http://gradle.org/ > Bye, > > Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng > upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik > > On Sat, Feb 20, 2016 at 7:02 AM, 'Ho, Ming' via SYSLOG-NG < > syslog-ng(a)balabit.com> wrote: > > > Hello, > > > > I am able to compile syslog-ng 3.7.2 without the Java-based modules > > included successfully and the syslog-ng run fine. Now I need to > > re-compile again with the Java-based modules included. I execute the > > following steps listed in the syslog-ng-3.7-ose-admin guide: > > > > Installed openjdk version "1.8.0_71" > > Installed gradle version 2.11 > > export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:$PKG_CONFIG_PATH > > export > > LD_LIBRARY_PATH=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.71-1.b15.el6_7 > > .x 86_64/jre/lib/amd64/server:$LD_LIBRARY_PATH > > > > When I run ./configure, I notice the java module option is set to "no" > > in the summary: > > Modules: > > Module search path : ${exec_prefix}/lib/syslog-ng > > Sun STREAMS support (module): no > > SQL support (module) : no > > PACCT module (EXPERIMENTAL) : no > > MongoDB destination (module): yes > > JSON support (module) : no > > SMTP support (module) : no > > AMQP destination (module) : yes > > STOMP destination (module) : yes > > GEOIP support (module) : no > > Redis support (module) : no > > Riemann destination (module): no > > python : yes (pkg-config package: python) > > java : no > > > > I go ahead to execute "make" and "make install" without error. But I > > don't see the "java-modules" directory is created in > > /usr/local/lib/syslog-ng directory. > > When I include the following declaration in my syslog-ng.conf to > > start syslog-ng, I got the following error: > > [2016-02-20T05:46:59.039466] Plugin module not found in > > 'module-path'; module-path='/usr/local/lib/syslog-ng', module='mod-java' > > > > Why Java modules is not included? What steps am I missing? > > Any help is appreciated > > > > Ming > > > > > > > > ____________________________________________________________________ > > __ > > ________ Member info: > > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > Documentation: > > http://www.balabit.com/support/documentation/?product=syslog-ng > > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > > > > > > > -------------- next part -------------- An HTML attachment was > scrubbed... > URL: > http://lists.balabit.hu/pipermail/syslog-ng/attachments/20160220/e2a92 > c57/attachment.html > > ------------------------------ > > Message: 2 > Date: Sat, 20 Feb 2016 11:24:24 +0100 > From: ?rp?d Magos?nyi <mag(a)magwas.rulez.org> > Subject: [syslog-ng] ECONNREFUSED on /dev/log > To: syslog-ng(a)lists.balabit.hu > Message-ID: <56C83ED8.4080305(a)magwas.rulez.org> > Content-Type: text/plain; charset=utf-8 > > Hi! > > "strace logger hello" output contains this line: > connect(3, {sa_family=AF_LOCAL, sun_path="/dev/log"}, 110) = -1 > ECONNREFUSED (Connection refused) > > I see logs in /dev/xconsole, just cannot emit a log line with logger. > What could be the cause? > > additional info: > > # cat /etc/syslog-ng/syslog-ng.conf > @version: 3.3 > @include "scl.conf" > # First, set some global options. > options { chain_hostnames(off); flush_lines(0); use-dns(persist_only); > dns-cache-hosts(/etc/hosts);use_fqdn(no); > owner("root"); group("adm"); perm(0640); stats_freq(0); > time_reap(1000); > bad_hostname("^gconfd$"); > }; > source s_src { > system(); > internal(); > }; > > #destination d_net { tcp("91.143.88.140" port(10000) > tls(ca-dir("/etc/ssl") cert-file("/etc/ssl/newcert.pem") > key_file("/etc/ssl/private/newkey.pem") )); }; destination d_net { tcp(" > infra.edemokraciagep.org" port(10000) > tls(ca-dir("/etc/ssl") peer-verify(optional-untrusted))); }; > > destination d_messages { file("/var/log/messages");}; > > destination d_xconsole { pipe("/dev/xconsole"); }; destination > d_apache_console { pipe("/dev/apacheconsole"); }; > > > filter apache_log { program("apache"); }; filter non_apache_log { not > program("apache"); }; > > destination d_vhost_gepnarancs { file("/var/log/vhost_gepnarancs"); }; > filter f_gepnarancs { program("apache:php:gepnarancs") or > message("gepnarancs"); }; log { source(s_src); filter(f_gepnarancs); > destination(d_vhost_gepnarancs); }; > > log { source(s_src); destination(d_messages); }; > > log { source(s_src); filter(non_apache_log) ; destination(d_xconsole); > }; > > log { source(s_src); filter(apache_log); > destination(d_apache_console); }; > > log { source(s_src); destination(d_net); }; > > @include "/etc/syslog-ng/conf.d/" > > --------------------------------- > # lsof |grep /dev/xconsole > syslog-ng 2608 root 15u FIFO 0,5 > 0t0 19564 /dev/xconsole > cat 2641 root 3r FIFO 0,5 > 0t0 19564 /dev/xconsole > --------------------------------- > # lsof |grep /dev/log > [no output] > --------------------------------- > # dpkg -l syslog-ng > Desired=Unknown/Install/Remove/Purge/Hold > | > > Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Tri > g-pend > |/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad) > ||/ Name > Version Architecture Description > +++-================================================-================= > +++== > +++==========-=============================-========================== > +++== > +++=================================================================== > +++== > +++===== > ii syslog-ng > 3.5.6-2 all Enhanced > system logging daemon (metapackage) > -------------------------------- > # ls -l /dev/log > srw-rw-rw- 1 root root 0 Feb 20 11:03 /dev/log > -------------------------------- > > > > ------------------------------ > > _______________________________________________ > syslog-ng maillist - syslog-ng(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/syslog-ng > > > End of syslog-ng Digest, Vol 130, Issue 27 > ****************************************** > >

1 0