November 2013 - syslog-ng - lists.balabit.hu

syslog-ng 3.4.5 has been released
by devel＠balabit.hu 08 Nov '13

08 Nov '13

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.4.5 SUMMARY : new stable release DATE : Nov 4, 2013 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.4.5) has been released. For latest fixes in the 3.4.x feature branch you are recommended to upgrade to this version. CHANGES: 3.4.5 Mon, 4 Nov 2013 08:54:50 +0100 This is the fourth bug-fix release for the 3.4.x series, with important fixes over the 3.4.4 release. Upgrading to this release is highly recommended. Bugfixes ======== * A race condition in log message reference counting code that sometimes led to crashes was fixed. [#255] * An assertion that happened in certain situations when a subst() rewrite rule was used was fixed. [#256] * A use-after-free error that sometimes happened after a reload, and caused memory corruption was also fixed. [#253] * patterndb was corrected not to create a new instance on reload: this way, the old one is not leaked, and db-parser() does not forget the correlation state, nor its idea of current time on reload. [#257] * The syslog-ng.spec file does not try to install the long-removed ChangeLog file anymore. * A few harmless, but noisy compiler warnings have been eliminated. Features ======== * The JSON parser now understands NULL-typed keys, and ignores them. Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Balazs Scheidler <bazsi(a)balabit.hu> Evan Rempel <erempel(a)uvic.ca> Gergely Nagy <algernon(a)balabit.hu> Gonzalo Paniagua <gonzalo.paniagua+slng1(a)acquia.com> Jose Pedro Oliveira <jpo(a)di.uminho.pt> Michal Privoznik <miso.privoznik(a)gmail.com> Nicolas Szalay <nico(a)rottenbytes.info> DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

2 1

syslog-ng Message Deliver Acknowledgment and Action
by Tamayo, Andres 07 Nov '13

07 Nov '13

Hello, I am new at using syslog-ng and I was hoping someone in the developers mailing list would be able to help me. I am trying to use a syslog-ng client daemon and a syslog-ng server daemon to implement an audit message system but I cannot find information that would help me configure the daemons for my particular scenario. Here is my scenario: I have a separate audit daemon that generates log messages that are written to an audit log file in syslog format. I need the syslog-ng client to read the logs in the audit log file and send them to the syslog-ng server. When the server has received the messages, I need some acknowledgment from the syslog-ng client, so my other audit daemon can remove the submitted log messages from the audit log file (preventing it from reaching maximum capacity). My scenario does not have to be setup exactly this way if there are better ways to achieve the same result. Basically, I need to keep all logs that have not yet being sent to the syslog-ng server in an audit file. When the messages are delivered to the syslog-ng server, I need to delete them from the audit file. Can someone tell me if I can achieve this results using syslog-ng daemons? Is there a better way to implement my scenario. Thank you for your help in advance.

2 2

Debian & Ubuntu packages for syslog-ng
by Gergely Nagy 06 Nov '13

06 Nov '13

As a reminder, and following Peter's FreeBSD & SuSE package news, for our Debian or Ubuntu users, I'm hosting syslog-ng 3.3, 3.4 and 3.5 packages for a wide range of distributions (see below), available at: http://asylum.madhouse-project.org/projects/debian/ syslog-ng 3.3 packages are available for Ubuntu Lucid, Precise, Raring, and Saucy (for both i386 and amd64 architectures) along with Debian Squeeze, Wheezy, Jessie and unstable (i386, amd64 and powerpc architectures). The syslog-ng 3.4 and 3.5 packages are also available for Ubuntu Trusty. The syslog-ng 3.3 package will be maintained for all of these platforms for as long as the 3.3 branch is maintained (and Ubuntu Trusty will be added to the list if and when 3.3.12 gets released). Same goes for 3.4 and 3.5: they'll be maintained until their respective end of life (see my earlier post about that[1]). I will keep adding new Ubuntu and Debian releases as they come out, and remove them shortly after they reach their respective end of lives. [1]: http://asylum.madhouse-project.org/blog/2013/10/22/syslog-ng-eol-dates/ The easiest way to use the repository is to use the interface on the website to select the distribution and components, then copy & paste to /etc/apt/sources.list. But the line to copy follows this pattern: deb http://packages.madhouse-project.org/debian wheezy syslog-ng-3.4 Where the end of the URL is either "debian" or "ubuntu", the release is your release (either the codename, such as "precise" or "wheezy", or version number like "12.04" or "7.0"), and the last part, the component is the syslog-ng branch you want. There is also a syslog-ng and syslog-ng-devel component, which will always follow the currently recommended branches (3.4 and 3.5 at the time of this writing), so if you want to update from one stable version to the other without having to modify your sources.list, I recommend using one of these aliases. A quick and easy way to enable the repository is as follows (as root): # curl https://packages.madhouse-project.org/debian/archive-key.txt | apt-key add - # echo "deb http://packages.madhouse-project.org/$(lsb_release -is | tr '[:upper:]' '[:lower:]') " \ "$(lsb_release -cs | tr '[:upper:]' '[:lower:]') " \ "syslog-ng" \ > /etc/apt/sources.list.d/syslog-ng.list -- |8]

1 0

git repository reorganization
by Balazs Scheidler 06 Nov '13

06 Nov '13

Hi, Until now, we have had separate git repositories for syslog-ng on github, one for each major version. This seemed logical at the time, since we do split maintenance, Algernon is doing the maintenance for older trees, whereas I maintain the latest version. Also, this is how I used to do it on my internal trees. However, this complicates things for people who want to follow syslog-ng development, whenever we create a new major release, they have to follow a different repository. Difficult to track forks, see all releases in one place, etc. Since git is perfectly capable of storing multiple major releases in the same git repository, I think it would be the best to let the multiple repositories idea go and stick to a single repository model in the future. To model this idea, I have created this repository on github, and pushed 3.3, 3.4 and 3.5 HEADs here: https://github.com/balabit/syslog-ng Branch names would be as follows: - master: the latest HEAD where development is happening on the latest release - 3.3/master, 3.4/master, 3.5/master: the HEAD for maintained old major releases - topic branches destined for a specific major release would be named 3.X/<branch>, topic branches for the latest version would omit the version number prefix. - tags indicate releases, they are named after the respective release for example: v3.4.5 I hope this makes things simpler for everyone in the long term. If anyone has feedback, I'd love to hear them now, before announcing this to the general public and update references everywhere. Thanks in advance, -- Bazsi

3 3

syslog-ng packages for openSUSE/SLES
by Peter Czanik 06 Nov '13

06 Nov '13

Hello, I just updated the syslog-ng packages in my openSUSE Build Service home project. As not all dependencies are part of SLES or openSUSE, it is recommended to add repositories by "zypper ar http://to/repo reponame" and install from there as my repositories carry all necessary dependencies as well. https://build.opensuse.org/project/show/home:czanik:syslog-ng33 has syslog-ng 3.3.11 and will be removed as soon as openSUSE 12.2 is EoL (early next year) https://build.opensuse.org/project/show/home:czanik:syslog-ng34 has syslog-ng 3.4.5 and will stay as long as openSUSE 12.3 is supported (Q3 next year) This version will be published as an update for both openSUSE 12.3 and 13.1 in the coming weeks (as it's not a security update, review is slower by the maintenance team) https://build.opensuse.org/project/show/home:czanik:syslog-ng35 has syslog-ng 3.5.1. This version adds redis support. Due to an error in the release process, manual pages are missing, but documentation is available at the usual location: http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-3.5-guid… As these are not official Balabit or SuSE packages, please contact me, if you run into any troubles! Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

[Bug 259] New: udp destination causes seg fault when BUILD_WITH_SSL is not defined
by bugzilla＠bugzilla.balabit.com 06 Nov '13

06 Nov '13

https://bugzilla.balabit.com/show_bug.cgi?id=259 Summary: udp destination causes seg fault when BUILD_WITH_SSL is not defined Product: syslog-ng Version: 3.5.x Platform: Other OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: htsai(a)skytap.com Type of the Report: bug Estimated Hours: 0.0 We ran into this bug when compiling 3.5.0rc1 on top of customized OpenWrt. BUILD_WITH_SSL is not defined on our platform. When a udp destination is defined, the start of syslog-ng failed with seg fault. After debugging, we found the bug is that, in afinet_dd_new_instance (afinet-dest.c:509), AFSocketDestDriver->construct_transport is initialized with afinet_dd_construct_transport, which is defined as afsocket_dd_construct_transport (line 158). When afsocket_dd_construct_transport is called to construct the transport (afsocket-dest.c:166), it calls AFSocketDestDriver->construct_transport (afsocket-dest.h:66), which is again afsocket_dd_construct_transport. The two functions call each other endlessly until the stack is exhausted, hence the seg fault. The bug still exists in 3.5.1 and may potentially affect other destinations using afinet. We didn't check other syslog-ng versions. The bug can be fixed by the following patch. Please review and apply the patch. --- a/modules/afsocket/afinet-dest.c +++ b/modules/afsocket/afinet-dest.c @@ -155,7 +155,7 @@ afinet_dd_construct_transport(AFSocketDe #else -#define afinet_dd_construct_transport afsocket_dd_construct_transport +#define afinet_dd_construct_transport afsocket_dd_construct_transport_method #endif -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 1

syslog-ng in FreeBSD ports
by Peter Czanik 06 Nov '13

06 Nov '13

Hello, Thanks to Cy of FreeBSD project, syslog-ng in the FreeBSD ports got updated in the past few days. There are multiple syslog-ng ports, here is a list, which version they carry now: sysutils/syslog-ng-devel: this is where all the fun happens, the latest and greatest syslog-ng, right now 3.5.1. It has redis support added, but no manual pages (missing from the release tgz). sysutils/syslog-ng: the "main" syslog-ng port has now 3.4.5 and considered to be the best balance between new features and maturity sysutils/syslog-ng33: the 3.3 line is still available, currently at 3.3.11 for those willing to stay with 3.3 sysutils/syslog-ng2: has the long time EoL 2.0.10, which I plan to remove as soon as all major Linux distributions stopped using it (when migrating to FreeBSD, it's easier to use the same version first and update later) Most likely around the time syslog-ng 3.6 beta is released, the current syslog-ng port will be copied to syslog-ng34, syslog-ng-devel to syslog-ng and syslog-ng-devel will be updated to 3.6. FYI: checking version is easy using FreshPorts: http://www.freshports.org/search.php?query=syslog-ng Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

[Bug 257] New: memory leak on reload
by bugzilla＠bugzilla.balabit.com 04 Nov '13

04 Nov '13

https://bugzilla.balabit.com/show_bug.cgi?id=257 Summary: memory leak on reload Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: --- Estimated Hours: 0.0 We have a complex environment that growns it's memory footprint on every reload. It is completely consistent as long as a reload is not done. A simple configuration does not have this problem. We have: - multiple patterndb uses (4 different databases) - multiple program destinations with different templates - multiple pipe destinations - multiple file destinations Every reload the memory difference is different amount and the difference does not always increase or decrease. For me it is about 13MB every reload. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 17

syslog-ng 3.5.1 has been released
by devel＠balabit.hu 04 Nov '13

04 Nov '13

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.5.1 SUMMARY : new stable release DATE : Nov 4, 2013 ------------------------------------------------------------------------------ DESCRIPTION: The first stable version of syslog-ng Open Source Edition (3.5.1) has been released. For the latest features in syslog-ng, you are encouraged to upgrade to this version. CHANGES: 3.5.1 Mon, 4 Nov 2013 14:49:00 +0100 This is the first stable release in the 3.5 series, adding a number of features compared to 3.4, a result of about eight months of development. This release includes all the fixes of the recent 3.4.5 release, and a host of new features. Bugfixes since 3.5.0rc1 ======================= * A race condition in log message reference counting code that sometimes led to crashes was fixed. [#255] * A use-after-free error that sometimes happened after a reload, and caused memory corruption was also fixed. [#253] * patterndb was corrected not to create a new instance on reload: this way, the old one is not leaked, and db-parser() does not forget the correlation state, nor its idea of current time on reload. [#257] * The syslog-ng.spec file does not try to install the long-removed ChangeLog file anymore. Incompatible changes ==================== * Template escaping was changed in an incompatible way: previously, both the lexer and the template compiler used the '\' character for escaping, which was confusing. The template compiler uses '$$' to escape the '$' char, and '@@' to escape '@'. If a non-numeric value follows '@', a warning will be printed. * The replace() transformation function of value-pairs() was renamed to replace-prefix() to better reflect what it actually does. The old name remains as an obsolete alias, however. * The username() and password() options of the MongoDB destination were removed, they never did anything before, either. Major features since 3.4 ======================== Multi-line support ------------------ A major feature in the 3.5 release is the inclusion of support for multi-line messages, a feature that has been available in syslog-ng PE for a good while. With this, both indented and regexp-based multi-line support becomes available. For further information and examples, see the following blog post: http://asylum.madhouse-project.org/blog/2013/09/05/what-is-cooking-in-syslo… STOMP destination ----------------- Joining the message-queue club, this new destination makes syslog-ng able to send events to any STOMP-capable message queuing server, such as Apache ActiveMQ. For further information about the destination, see this commit message: https://github.com/balabit/syslog-ng-3.5/commit/82d19a4d4ad3f91e8214006f6ea… Redis destination ----------------- Developed during the Google Summer of Code 2013 program, this destination driver makes it possible to easily send commands to a Redis server. For further information and examples, please see the following blog post: http://tichygsoc.blogspot.hu/2013/09/the-road-so-far.html Template type hinting --------------------- While syslog-ng supported sending events to various datastores and queues for a while now (SQL first, MongoDB, JSON, and AMQP later), even if those supported other types of data, syslog-ng only ever sent strings. With template type hinting, it became possible to tell syslog-ng what type a certain template should be, so that the drivers can use that information in whatever way they see fit. This is currently implemented for the MongoDB driver and the $(format-json) template function only. For more information about type hinting and for examples, see the following post: http://asylum.madhouse-project.org/blog/2013/09/05/what-is-cooking-in-syslo… Template options honored everywhere ----------------------------------- Until this release, there were situations where template options were ignored, such as filter expressions that use the comparsion operators, regexp based substitutions, incoming templates for parsers, the new value rewrite rules, SMTP values, some of the new drivers (stomp, amqp), and patterndb context-id. These all honor the global template options now, and per-driver options such as frac-digits and local-time-zone are available for drivers which did not have them before, like MongoDB. Support for unit suffixes in the configuration ---------------------------------------------- You no longer need to remember how many zeros to put after a big number in the syslog-ng configuration file, you can use unit suffixes, such as: log-fifo-size(2GiB) See the following post for more details: http://asylum.madhouse-project.org/blog/2013/09/05/what-is-cooking-in-syslo… The Incubator project --------------------- Alongside the 3.5.1 release, we are pleased to announce the existence of the syslog-ng incubator project, which hosts additional modules and tools not merged into syslog-ng proper. These serve both as examples and as a staging area, but also makes it easier to try out new modules without patching or upgrading your syslog-ng version. The project's homepage is: https://github.com/balabit/syslog-ng-incubator Other features ============== * in-list() filter: this new filter function allows one to easily implement file-based white- and blacklists with a simple syntax: filter f_white { in-list("/path/to/file", value("HOST")); }; * A set of new string-related template functions are available: $(uppercase STRINGS...), $(lowercase STRINGS...) and $(replace-delimiter DELIMITERS NEW-DELIMITER TEXT). The first two do exactly what their names suggest, while the last one replaces all occurrences of any DELIMITERS within TEXT with the NEW-DELIMITER. * There is also a new $(env VARIABLE...) template function, which looks up the given variables in the environment. This is similar to using backticks in the configuration file, however the name of the environment variable with $(env) may contain macros too. * Support for Linux 3.5+'s /dev/kmsg was added, and the system() source will automatically detect whether to use it over /proc/kmsg. * For every correlated message in patterndb, the ${.classifier.context_id} property is automatically set to the context-id attribute of the matching rule. * The build system was completely redone, it is much faster, more reliable, and less verbose by default now. * Several systemd-related enhancements were made, including support for notification-based startup. This also means that when systemd support is compiled in, libsystemd-daemon becomes a new dependency. Known Bugs ========== * The afstreams module is broken, does not compile, and does not work. This will be corrected in a later maintenance release. Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Alexandre Biancalana <biancalana(a)gmail.com> Andras Tim <tia(a)balabit.hu> Anton Koldaev <koldaevav(a)gmail.com> Attila M. Magyar <athos(a)balabit.hu> Attila Nagy <bra(a)fsn.hu> Attila Szalay <sasa(a)balabit.hu> Balazs Scheidler <bazsi(a)balabit.hu> Balint Kovacs <blint(a)balabit.hu> Chris Johnson <chris.johnson3(a)hp.com> Cy Schubert <Cy.Schubert(a)komquats.com> Evan Rempel <erempel(a)uvic.ca> Fabien Wernli <cpan(a)faxm0dem.org> Gergely Nagy <algernon(a)balabit.hu> Gonzalo Paniagua <gonzalo.paniagua+slng1(a)acquia.com> Jose Pedro Oliveira <jpo(a)di.uminho.pt> Laszlo Budai <lbudai(a)balabit.hu> Lucas McLane <lucas(a)clicksecurity.com> Marc Falzon <marc.falzon(a)cloudwatt.com> Martin <bmartin(a)lavabit.com> Michal Privoznik <miso.privoznik(a)gmail.com> Michael Sterrett <mr_bones_(a)gentoo.org> Nicolas Szalay <nico(a)rottenbytes.info> Oscar Muñoz Paul Dann <pdgiddie+balabit(a)gmail.com> Peter Czanik <czanik(a)balabit.hu> Peter Gyongyosi <gyp(a)balabit.hu> Robert Fekete <frobert(a)balabit.hu> Ryan Frederick Sergey Shuman Tamas Pal <folti(a)balabit.hu> Tibor Benke <btibi(a)balabit.hu> Tihamer Petrovics <tihameri(a)gmail.com> Valentijn Sessink <valentijn(a)sessink.nl> Viktor Juhasz <jviktor(a)balabit.hu> Viktor Tusa <tusa(a)balabit.hu> Vincent Brillault <spam(a)lerya.net> DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

[Bug 255] New: segmentation faults often
by bugzilla＠bugzilla.balabit.com 02 Nov '13

02 Nov '13

https://bugzilla.balabit.com/show_bug.cgi?id=255 Summary: segmentation faults often Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: blocker Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: bug Estimated Hours: 0.0 In my setup, which is big, using syslog-ng 3.4.4 I get segmentation faults 2-3 times per minute. 3.4.1 did this once every couple of weeks, and 3.4.2 was worse, but 3.4.4 is completely unusable for me. What would you like from me to aid tracking this down? -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

3 15