- syslog-ng - lists.balabit.hu

a log message is output in the two line
by Hirose, Shinsaku 30 Jun '16

30 Jun '16

Hello, all, I use syslog-ng-3.2.5-4.el6.x86_64 on Centos6. I got it from eple epel repository. I am troubled in how to use the source file driver. The touble is that a log message is output in the two lines on remote syslog server. How to reproduce is followings. 1. Prepare two hosts running syslog-ng. Host_A configuration is followings. ---------------------------------------- source test { file("/tmp/a.log"); }; destination d_remote { udp("192.168.0.2"); }; log { source(test); destination(d_remote); }; ---------------------------------------- Host_B(192.168.0.2) configuration is defaults. 2.Execute following command on Host_A. $ seq 8193 | (xargs -i echo -n "a";echo "") >> /tmp/a.log 3.Check the log on Host_B. As the result, a log message is output in the two lines on Host_B. One line is following. The num of "a" is 8192. aaaaaaaaaaaaa....... The other line is following. The num of "a" is 1. a I hope a log message is output in the one line on Host_B. Is my hope readily achievable? Please advise me.

5 7

CzP in Nuremberg
by Czanik, Péter 21 Jun '16

21 Jun '16

Hi, For the next couple of days I'll be in Nuremberg for the openSUSE conference: https://events.opensuse.org/conference/oSC16 I'll give a presentation on syslog-ng Friday in the afternoon at 13:30 in the Roter Salon. I'll wear my syslog-ng superhero t-shirt, so I'll be easy to spot and ready to answer syslog-ng questions any time. If you are in Nuremberg, but not at the conference, I'm happy to talk to you about syslog-ng also outside of the conference floor. You can reach me by e-mail or on twitter as @pczanik See you soon in Nuremberg, Peter

1 0

Syslog-ng 3.7.3 + Elasticsearch V2.3 - Error init pipeline
by Marco Mignone 16 Jun '16

16 Jun '16

Hi All, I am trying to setup syslog-ng to use elasticsearch as its destination on Ubuntu 14.04. This the version of syslog (the unofficial versions installed from laszlo_budai rep: syslog-ng 3.7.3 Installer-Version: 3.7.3 Revision: 3.7.3-8 Compile-Date: Jun 1 2016 16:33:00 Available-Modules: basicfuncs,linux-kmsg-format,riemann,afuser,afstomp,afprog,json-plugin,afsmtp,affile,csvparser,mod-java,pseudofile,confgen,afsocket,afamqp,redis,sdjournal,kvformat,syslogformat,afsql,system-source,mod-python,graphite,dbparser,geoip-plugin,afmongodb,cryptofuncs Elastic search is: "name" : "Theresa Cassidy", "cluster_name" : "elasticsearch", "version" : { "number" : "2.3.3", "build_hash" : "218bdf10790eef486ff2c41a3df5cfa32dadcfde", "build_timestamp" : "2016-05-17T15:40:04Z", "build_snapshot" : false, "lucene_version" : “5.5.0" And my custom configuration in /etc/syslog-ng/conf.d/test.conf which is: @module mod-java source s_net { udp(); tcp(); }; destination d_elastic { elasticsearch( index("syslog-ng_${YEAR}.${MONTH}.${DAY}") type("test") client_lib_dir("/usr/share/elasticsearch/lib") ); }; log { source(s_net); destination(d_elastic); flags(flow-control); }; When I try to launch syslog in debug mode this is what I get: [2016-06-16T15:54:29.378356] Add path to classpath: /usr/lib/syslog-ng/3.7/java-modules/syslog-ng-core.jar; [2016-06-16T15:54:29.382446] Add path to classpath: /usr/lib/syslog-ng/3.7/java-modules/syslog-ng-core.jar; [2016-06-16T15:54:29.382660] Add path to classpath: /usr/lib/syslog-ng/3.7/java-modules/kafka.jar; [2016-06-16T15:54:29.382862] Add path to classpath: /usr/lib/syslog-ng/3.7/java-modules/http.jar; [2016-06-16T15:54:29.383052] Add path to classpath: /usr/lib/syslog-ng/3.7/java-modules/hdfs.jar; [2016-06-16T15:54:29.383258] Add path to classpath: /usr/lib/syslog-ng/3.7/java-modules/elastic.jar; [2016-06-16T15:54:29.383479] Add path to classpath: /usr/lib/syslog-ng/3.7/java-modules/syslog-ng-common.jar; [2016-06-16T15:54:29.383670] Add path to classpath: /usr/lib/syslog-ng/3.7/java-modules/log4j-1.2.16.jar; [2016-06-16T15:54:29.383917] Add path to classpath: /usr/share/elasticsearch/lib/guava-18.0.jar; [2016-06-16T15:54:29.384098] Add path to classpath: /usr/share/elasticsearch/lib/jna-4.1.0.jar; [2016-06-16T15:54:29.384293] Add path to classpath: /usr/share/elasticsearch/lib/spatial4j-0.5.jar; [2016-06-16T15:54:29.384494] Add path to classpath: /usr/share/elasticsearch/lib/compress-lzf-1.0.2.jar; [2016-06-16T15:54:29.386104] Add path to classpath: /usr/share/elasticsearch/lib/lucene-spatial-5.5.0.jar; [2016-06-16T15:54:29.386342] Add path to classpath: /usr/share/elasticsearch/lib/lucene-backward-codecs-5.5.0.jar; [2016-06-16T15:54:29.386507] Add path to classpath: /usr/share/elasticsearch/lib/t-digest-3.0.jar; [2016-06-16T15:54:29.386677] Add path to classpath: /usr/share/elasticsearch/lib/jackson-dataformat-cbor-2.6.6.jar; [2016-06-16T15:54:29.386865] Add path to classpath: /usr/share/elasticsearch/lib/lucene-memory-5.5.0.jar; [2016-06-16T15:54:29.387044] Add path to classpath: /usr/share/elasticsearch/lib/jackson-core-2.6.6.jar; [2016-06-16T15:54:29.387216] Add path to classpath: /usr/share/elasticsearch/lib/joda-time-2.8.2.jar; [2016-06-16T15:54:29.387394] Add path to classpath: /usr/share/elasticsearch/lib/lucene-join-5.5.0.jar; [2016-06-16T15:54:29.387673] Add path to classpath: /usr/share/elasticsearch/lib/lucene-grouping-5.5.0.jar; [2016-06-16T15:54:29.388476] Add path to classpath: /usr/share/elasticsearch/lib/HdrHistogram-2.1.6.jar; [2016-06-16T15:54:29.388647] Add path to classpath: /usr/share/elasticsearch/lib/compiler-0.8.13.jar; [2016-06-16T15:54:29.388818] Add path to classpath: /usr/share/elasticsearch/lib/netty-3.10.5.Final.jar; [2016-06-16T15:54:29.388972] Add path to classpath: /usr/share/elasticsearch/lib/lucene-misc-5.5.0.jar; [2016-06-16T15:54:29.389518] Add path to classpath: /usr/share/elasticsearch/lib/lucene-analyzers-common-5.5.0.jar; [2016-06-16T15:54:29.389711] Add path to classpath: /usr/share/elasticsearch/lib/lucene-sandbox-5.5.0.jar; [2016-06-16T15:54:29.390094] Add path to classpath: /usr/share/elasticsearch/lib/elasticsearch-2.3.3.jar; [2016-06-16T15:54:29.390283] Add path to classpath: /usr/share/elasticsearch/lib/commons-cli-1.3.1.jar; [2016-06-16T15:54:29.390488] Add path to classpath: /usr/share/elasticsearch/lib/snakeyaml-1.15.jar; [2016-06-16T15:54:29.390659] Add path to classpath: /usr/share/elasticsearch/lib/jsr166e-1.1.0.jar; [2016-06-16T15:54:29.390935] Add path to classpath: /usr/share/elasticsearch/lib/lucene-core-5.5.0.jar; [2016-06-16T15:54:29.391176] Add path to classpath: /usr/share/elasticsearch/lib/lucene-suggest-5.5.0.jar; [2016-06-16T15:54:29.394616] Add path to classpath: /usr/share/elasticsearch/lib/jackson-dataformat-yaml-2.6.6.jar; [2016-06-16T15:54:29.395279] Add path to classpath: /usr/share/elasticsearch/lib/log4j-1.2.17.jar; [2016-06-16T15:54:29.395458] Add path to classpath: /usr/share/elasticsearch/lib/joda-convert-1.2.jar; [2016-06-16T15:54:29.395970] Add path to classpath: /usr/share/elasticsearch/lib/hppc-0.7.1.jar; [2016-06-16T15:54:29.396734] Add path to classpath: /usr/share/elasticsearch/lib/jackson-dataformat-smile-2.6.6.jar; [2016-06-16T15:54:29.397919] Add path to classpath: /usr/share/elasticsearch/lib/lucene-queryparser-5.5.0.jar; [2016-06-16T15:54:29.398106] Add path to classpath: /usr/share/elasticsearch/lib/lucene-spatial3d-5.5.0.jar; [2016-06-16T15:54:29.398281] Add path to classpath: /usr/share/elasticsearch/lib/lucene-queries-5.5.0.jar; [2016-06-16T15:54:29.398440] Add path to classpath: /usr/share/elasticsearch/lib/apache-log4j-extras-1.2.17.jar; [2016-06-16T15:54:29.398610] Add path to classpath: /usr/share/elasticsearch/lib/lucene-highlighter-5.5.0.jar; [2016-06-16T15:54:29.398784] Add path to classpath: /usr/share/elasticsearch/lib/jts-1.13.jar; [2016-06-16T15:54:29.398925] Add path to classpath: /usr/share/elasticsearch/lib/securesm-1.0.jar; [2016-06-16T15:54:29.501879] Add path to classpath: /usr/lib/syslog-ng/3.7/java-modules/syslog-ng-core.jar; [2016-06-16T15:54:29.519443] Error initializing message pipeline; Was anyone able to make this work or suggest a way to fix this? I hope I am on the right place and if not apologies in advance. Thanks, Marco

3 3

Re: [syslog-ng] Syslog-ng 3.7.3 + Elasticsearch V2.3 - Error init pipeline
by Laszlo Budai 16 Jun '16

16 Jun '16

Hi, it should work with the latest 1.x. I'm working on my 3.8 OBS repo, but having some problems. When I have the alpha packages, I'll send a mail. Regards, Laszlo Budai Sent from my HTC ----- Reply message ----- From: "Marco Mignone" <info(a)marcomignone.com> To: "Syslog-ng users' and developers' mailing list" <syslog-ng(a)lists.balabit.hu> Subject: [syslog-ng] Syslog-ng 3.7.3 + Elasticsearch V2.3 - Error init pipeline Date: Thu, Jun 16, 2016 17:34 Hi Peter, Thanks for the quick reply. I thought there was some hack or way around it. I will then try with ElasticSearch1.X. Will the latest version of 1.x work or is there a specific version? Thanks again for your time. Marco > On 16 Jun 2016, at 16:26, Czanik, Péter <peter.czanik(a)balabit.com> wrote: > > Hi, > > syslog-ng 3.7 only works with Elasticsearch 1.X. For Elasticsearch 2.X > you need syslog-ng 3.8, which is still under development (alpha). > > Bye, > Peter Czanik (CzP) <peter.czanik(a)balabit.com> > Balabit / syslog-ng upstream > http://czanik.blogs.balabit.com/ > https://twitter.com/PCzanik > > > On Thu, Jun 16, 2016 at 5:11 PM, Marco Mignone <info(a)marcomignone.com> wrote: >> Hi All, >> I am trying to setup syslog-ng to use elasticsearch as its destination on >> Ubuntu 14.04. >> This the version of syslog (the unofficial versions installed from >> laszlo_budai rep: >> >> syslog-ng 3.7.3 >> Installer-Version: 3.7.3 >> Revision: 3.7.3-8 >> Compile-Date: Jun 1 2016 16:33:00 >> Available-Modules: >> basicfuncs,linux-kmsg-format,riemann,afuser,afstomp,afprog,json-plugin,afsmtp,affile,csvparser,mod-java,pseudofile,confgen,afsocket,afamqp,redis,sdjournal,kvformat,syslogformat,afsql,system-source,mod-python,graphite,dbparser,geoip-plugin,afmongodb,cryptofuncs >> >> Elastic search is: >> >> "name" : "Theresa Cassidy", >> "cluster_name" : "elasticsearch", >> "version" : { >> "number" : "2.3.3", >> "build_hash" : "218bdf10790eef486ff2c41a3df5cfa32dadcfde", >> "build_timestamp" : "2016-05-17T15:40:04Z", >> "build_snapshot" : false, >> "lucene_version" : “5.5.0" >> >> >> And my custom configuration in /etc/syslog-ng/conf.d/test.conf which is: >> >> @module mod-java >> >> source s_net { >> udp(); >> tcp(); >> }; >> >> destination d_elastic { >> elasticsearch( >> index("syslog-ng_${YEAR}.${MONTH}.${DAY}") >> type("test") >> client_lib_dir("/usr/share/elasticsearch/lib") >> ); >> }; >> >> log { >> source(s_net); >> destination(d_elastic); >> flags(flow-control); >> }; >> >> >> When I try to launch syslog in debug mode this is what I get: >> >> [2016-06-16T15:54:29.378356] Add path to classpath: >> /usr/lib/syslog-ng/3.7/java-modules/syslog-ng-core.jar; >> [2016-06-16T15:54:29.382446] Add path to classpath: >> /usr/lib/syslog-ng/3.7/java-modules/syslog-ng-core.jar; >> [2016-06-16T15:54:29.382660] Add path to classpath: >> /usr/lib/syslog-ng/3.7/java-modules/kafka.jar; >> [2016-06-16T15:54:29.382862] Add path to classpath: >> /usr/lib/syslog-ng/3.7/java-modules/http.jar; >> [2016-06-16T15:54:29.383052] Add path to classpath: >> /usr/lib/syslog-ng/3.7/java-modules/hdfs.jar; >> [2016-06-16T15:54:29.383258] Add path to classpath: >> /usr/lib/syslog-ng/3.7/java-modules/elastic.jar; >> [2016-06-16T15:54:29.383479] Add path to classpath: >> /usr/lib/syslog-ng/3.7/java-modules/syslog-ng-common.jar; >> [2016-06-16T15:54:29.383670] Add path to classpath: >> /usr/lib/syslog-ng/3.7/java-modules/log4j-1.2.16.jar; >> [2016-06-16T15:54:29.383917] Add path to classpath: >> /usr/share/elasticsearch/lib/guava-18.0.jar; >> [2016-06-16T15:54:29.384098] Add path to classpath: >> /usr/share/elasticsearch/lib/jna-4.1.0.jar; >> [2016-06-16T15:54:29.384293] Add path to classpath: >> /usr/share/elasticsearch/lib/spatial4j-0.5.jar; >> [2016-06-16T15:54:29.384494] Add path to classpath: >> /usr/share/elasticsearch/lib/compress-lzf-1.0.2.jar; >> [2016-06-16T15:54:29.386104] Add path to classpath: >> /usr/share/elasticsearch/lib/lucene-spatial-5.5.0.jar; >> [2016-06-16T15:54:29.386342] Add path to classpath: >> /usr/share/elasticsearch/lib/lucene-backward-codecs-5.5.0.jar; >> [2016-06-16T15:54:29.386507] Add path to classpath: >> /usr/share/elasticsearch/lib/t-digest-3.0.jar; >> [2016-06-16T15:54:29.386677] Add path to classpath: >> /usr/share/elasticsearch/lib/jackson-dataformat-cbor-2.6.6.jar; >> [2016-06-16T15:54:29.386865] Add path to classpath: >> /usr/share/elasticsearch/lib/lucene-memory-5.5.0.jar; >> [2016-06-16T15:54:29.387044] Add path to classpath: >> /usr/share/elasticsearch/lib/jackson-core-2.6.6.jar; >> [2016-06-16T15:54:29.387216] Add path to classpath: >> /usr/share/elasticsearch/lib/joda-time-2.8.2.jar; >> [2016-06-16T15:54:29.387394] Add path to classpath: >> /usr/share/elasticsearch/lib/lucene-join-5.5.0.jar; >> [2016-06-16T15:54:29.387673] Add path to classpath: >> /usr/share/elasticsearch/lib/lucene-grouping-5.5.0.jar; >> [2016-06-16T15:54:29.388476] Add path to classpath: >> /usr/share/elasticsearch/lib/HdrHistogram-2.1.6.jar; >> [2016-06-16T15:54:29.388647] Add path to classpath: >> /usr/share/elasticsearch/lib/compiler-0.8.13.jar; >> [2016-06-16T15:54:29.388818] Add path to classpath: >> /usr/share/elasticsearch/lib/netty-3.10.5.Final.jar; >> [2016-06-16T15:54:29.388972] Add path to classpath: >> /usr/share/elasticsearch/lib/lucene-misc-5.5.0.jar; >> [2016-06-16T15:54:29.389518] Add path to classpath: >> /usr/share/elasticsearch/lib/lucene-analyzers-common-5.5.0.jar; >> [2016-06-16T15:54:29.389711] Add path to classpath: >> /usr/share/elasticsearch/lib/lucene-sandbox-5.5.0.jar; >> [2016-06-16T15:54:29.390094] Add path to classpath: >> /usr/share/elasticsearch/lib/elasticsearch-2.3.3.jar; >> [2016-06-16T15:54:29.390283] Add path to classpath: >> /usr/share/elasticsearch/lib/commons-cli-1.3.1.jar; >> [2016-06-16T15:54:29.390488] Add path to classpath: >> /usr/share/elasticsearch/lib/snakeyaml-1.15.jar; >> [2016-06-16T15:54:29.390659] Add path to classpath: >> /usr/share/elasticsearch/lib/jsr166e-1.1.0.jar; >> [2016-06-16T15:54:29.390935] Add path to classpath: >> /usr/share/elasticsearch/lib/lucene-core-5.5.0.jar; >> [2016-06-16T15:54:29.391176] Add path to classpath: >> /usr/share/elasticsearch/lib/lucene-suggest-5.5.0.jar; >> [2016-06-16T15:54:29.394616] Add path to classpath: >> /usr/share/elasticsearch/lib/jackson-dataformat-yaml-2.6.6.jar; >> [2016-06-16T15:54:29.395279] Add path to classpath: >> /usr/share/elasticsearch/lib/log4j-1.2.17.jar; >> [2016-06-16T15:54:29.395458] Add path to classpath: >> /usr/share/elasticsearch/lib/joda-convert-1.2.jar; >> [2016-06-16T15:54:29.395970] Add path to classpath: >> /usr/share/elasticsearch/lib/hppc-0.7.1.jar; >> [2016-06-16T15:54:29.396734] Add path to classpath: >> /usr/share/elasticsearch/lib/jackson-dataformat-smile-2.6.6.jar; >> [2016-06-16T15:54:29.397919] Add path to classpath: >> /usr/share/elasticsearch/lib/lucene-queryparser-5.5.0.jar; >> [2016-06-16T15:54:29.398106] Add path to classpath: >> /usr/share/elasticsearch/lib/lucene-spatial3d-5.5.0.jar; >> [2016-06-16T15:54:29.398281] Add path to classpath: >> /usr/share/elasticsearch/lib/lucene-queries-5.5.0.jar; >> [2016-06-16T15:54:29.398440] Add path to classpath: >> /usr/share/elasticsearch/lib/apache-log4j-extras-1.2.17.jar; >> [2016-06-16T15:54:29.398610] Add path to classpath: >> /usr/share/elasticsearch/lib/lucene-highlighter-5.5.0.jar; >> [2016-06-16T15:54:29.398784] Add path to classpath: >> /usr/share/elasticsearch/lib/jts-1.13.jar; >> [2016-06-16T15:54:29.398925] Add path to classpath: >> /usr/share/elasticsearch/lib/securesm-1.0.jar; >> [2016-06-16T15:54:29.501879] Add path to classpath: >> /usr/lib/syslog-ng/3.7/java-modules/syslog-ng-core.jar; >> [2016-06-16T15:54:29.519443] Error initializing message pipeline; >> >> >> Was anyone able to make this work or suggest a way to fix this? >> I hope I am on the right place and if not apologies in advance. >> >> Thanks, >> Marco >> >> >> ______________________________________________________________________________ >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >> Documentation: >> http://www.balabit.com/support/documentation/?product=syslog-ng >> FAQ: http://www.balabit.com/wiki/syslog-ng-faq >> >> > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

2 1

Is there a standard for naming tag/value pairs when parsing
by Evan Rempel 16 Jun '16

16 Jun '16

There was a project by Mitre (https://www.mitre.org/) called the Common Event Expression (https://cee.mitre.org/) that was going to be the official standard for metadata names for events, but that project has been stopped. Other than the two references that the CEE project has for logging standardization efforts, does anyone know of any major efforts by any group to define a standard for metadata naming? Evan.

3 6

Insider 2016-06: Red Hat; 3.8 RPMs; BMW; MongoDB; FLARE; Java;
by Czanik, Péter 16 Jun '16

16 Jun '16

Dear syslog-ng users, This is the 50th issue of syslog-ng Insider, a monthly newsletter that brings you syslog-ng-related news. NEWS Meet syslog-ng at Red Hat Summit -------------------------------- This year Balabit is sponsoring the Red Hat Summit again. The event will be held in Moscone Center North in San Francisco, June 27-29. Visit Balabit’s booth (booth #918) and meet Balázs Scheidler, the author of syslog-ng. You can request an appointment at http://pages2.balabit.com/red-hat-summit-appointment-request/ If you want to meet him outside of the conference floor, there will also be a meetup in Café Prague on the 30th of June: https://www.eventbrite.com/e/techie-happy-hour-optimize-your-log-data-in-th… Preview syslog-ng 3.8 RPM packages and send us your feedback ------------------------------------------------------------ While syslog-ng 3.8 does not yet have an alpha release, it already has many interesting features. As it is still under heavy development, we can't recommend it for production use. On the other hand, any feedback is very welcome. New features include disk buffer, Elasticsearch 2 support, grouping-by, Rust-based parsers, and so on: https://czanik.blogs.balabit.com/2016/05/state-of-syslog-ng-3-8-rpm-packagi… syslog-ng in the BMW i3 ----------------------- While most people know syslog-ng as a central syslog server, there is another use, which is less known but most likely has a lot larger installed base. It is syslog-ng embedded. Read how BMW is utilizing syslog-ng at https://czanik.blogs.balabit.com/2016/06/embedded-syslog-ng-bmw-i3-all-elec… and let us know how you use syslog-ng! MongoDB changes in 3.8 ---------------------- Under the hood, the driver to access MongoDB databases was replaced in syslog-ng 3.8. Read about the advantages of the new driver and how it affects configuration at: https://syslog-ng.org/mongodb-destination-receives-face-lift/ FLARE: Filtered Log Alert and Reporting Engine ---------------------------------------------- FLARE staplies together syslog-ng, ELK, Alerting and Incident Response with metadata. The aim was to develop a solution that is able to provide a unified view of log events and incidents, helping the work of operations admins with actionable insight. It is an internal software developed at the University of Victoria by long-time members of the syslog-ng community. You can read the complete presentation about this syslog-ng & PatternDB-based solution at https://www.bc.net/flare-stapling-together-syslog-ng-elk-alerting-and-incid… Writing syslog-ng Java destination drivers ------------------------------------------ If a destination is not supported by syslog-ng out of the box, you can develop a new driver in C or one of the language bindings. Here are the first steps for creating a Java-based destination driver: https://vithulanmv.wordpress.com/2016/06/07/the-syslog-ng-java-destinations/ Your feedback and news, or tips about the next issue are welcome at documentation(a)balabit.com . To read this newsletter online, visit: http://insider.blogs.balabit.com/ Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

Custom plugin/module
by Dmitry Shmulevich 16 Jun '16

16 Jun '16

Hello, I'm looking for an advice. I'm sending the logs from my machine to a third-party server. The protocol between the client (syslog-ng service) and the third-party server is quite simple: for each received log entry the server replies with a confirmation message on the same tcp socket. Currently I'm using a simple syslog-ng config to send logs via network driver to a specified ip/port. However, since syslog-ng service doesn't receive/read the messages from the server, the connection eventually dies. Is there any way to modify syslog-ng config to allow reads after writes? I didn't find anything similar in the documentation. An alternative way seems to be implementing a syslog-ng module or a plugin. Would that be the right approach? It seems that I can customize _LogProtoClient structure, and create a plugin. But I have no clue on how the overall API looks like, and what are the steps to build the module. Could you guys point me to the right resources? Is there any sample plugin, or a guidance for developing custom plugins? Thank you in advance, Dmitry

3 3

Embedded syslog-ng: BMW i3 all-electric car
by Czanik, Péter 13 Jun '16

13 Jun '16

Hi, A few years ago I wrote about syslog-ng running on all Kindle e-readers. Recently I found, that BMW also uses syslog-ng in the i3 all-electric car: https://czanik.blogs.balabit.com/2016/06/embedded-syslog-ng-bmw-i3-all-elec… If you have another interesting use case, either embedded or the more traditional central server, I'd be glad to add it to our "Powered by syslog-ng" page at https://syslog-ng.org/powered-by-syslog-ng/ Bye, Peter Czanik (CzP) <peter.czanik(a)balabit.com> Balabit / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

Re: [syslog-ng] Is there a standard for naming tag/value pairs when parsing
by jrhendri 12 Jun '16

12 Jun '16

There is something to be said for making it easy to use syslog-ng with the major players and right now that is splunk. I would hasten to add keeping json actively used and the native elasticsearch destination current also makes it easy to position syslog-ng as a standard collection and distribution layer. Within 2 to 3 years something will leapfrog splunk and if syslog-ng is ready, it will make that transition much easier. Jim Sent from my Verizon, Samsung Galaxy smartphone -------- Original message --------From: "Scheidler, Balázs" <balazs.scheidler(a)balabit.com> Date: 6/12/16 5:44 AM (GMT-05:00) To: Syslog-ng users' and developers' mailing list <syslog-ng(a)lists.balabit.hu> Subject: Re: [syslog-ng] Is there a standard for naming tag/value pairs when parsing Well, CEE is pretty much dead, and I didn't see too much activity wrt lumberjack either. I would rather see consolidation instead of further fragmentation in this area. Cheers Bazsi You are the last person I thought would point me toward the splunk CIM. Given the support that Balabit has put behind CEE and then lumberjack and even the experimental patternDB schema (https://github.com/balabit/syslog-ng-patterndb/blob/master/SCHEMAS.txt) I was sure you would steer me toward lumberjack. At first glance the splunk CIM appears to be structured around and partially dependant on some of the data flows of the splunk product. I'll continue to review it but at this point I am still open to alternate suggestions. Evan. On 06/11/2016 11:45 AM, Scheidler, Balázs wrote: There's common information model at splunk or the field dictionary of CEF, of arcsight fame. I would probably use the splunk one, except if you plan to use arcsight at the end. On Jun 11, 2016 18:32, "Evan Rempel" <erempel(a)uvic.ca> wrote: There was a project by Mitre (https://www.mitre.org/) called the Common Event Expression (https://cee.mitre.org/) that was going to be the official standard for metadata names for events, but that project has been stopped. Other than the two references that the CEE project has for logging standardization efforts, does anyone know of any major efforts by any group to define a standard for metadata naming? Evan. ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

1 0

UTF-8 handling
by Robin Blanchard 10 Jun '16

10 Jun '16

Hi, I've got syslog-ng-3.7.3-3.el7.centos.x86_64 feeding logstash-2.2.4-1.noarch. I've had flags(sanitize-utf8) enabled on the relevant syslog-ng listener, yet LS continues to complain with ':expected_charset=>"UTF-8"', to the point of LS ultimately choking. The underlying messages are indeed garbage, eg: $ pcregrep -n "[\x80-\xFF]" garbage.txt |tail -1 299:\\xc2\u0003\u0003\\xe0\u0001|\\x9d\\x9bX\\xa3wt}\\xe4\u001e\\xd6g\u0013j\\x9c!j\\xd9#*\\x95#\\xf8�� Should I instead be employing flags(validate-utf8) ? Ultimately, I'd like for syslog-ng to either re-encode messages into proper UTF8, or -- if not possible -- just squelch/drop them…. Thanks in advance -------------------------------------------------------------------------------------------------------------------------- This email has been sent to you on behalf of Nephila Advisors LLC (“Advisors”). Advisors provides consultancy services to Nephila Capital Ltd. (“Capital”), an investment advisor managed and carrying on business in Bermuda. Advisors and its employees do not act as agents for Capital or the funds it advises and do not have the authority to bind Capital or such funds to any transaction or agreement. The information in this e-mail, and any attachment therein, is confidential and for use by the addressee only. Any use, disclosure, reproduction, modification or distribution of the contents of this e-mail, or any part thereof, other than by the intended recipient, is strictly prohibited. If you are not the intended recipient, please return the e-mail to the sender and delete it from your computer. This email is for information purposes only, nothing contained herein constitutes an offer to sell or buy securities, as such an offer may only be made from a properly authorized offering document. Although Nephila attempts to sweep e-mail and attachments for viruses, it does not guarantee that either are virus-free and accepts no liability for any damage sustained as a result of viruses. --------------------------------------------------------------------------------------------------------------------------

1 0