- syslog-ng - lists.balabit.hu

insider 2014-01: Conferences; 3.6 pre-alpha ; incubator ; PCI-DSS
by Peter Czanik 09 Jan '14

09 Jan '14

Dear syslog-ng users, This is the 30th issue of the syslog-ng Insider, a monthly newsletter that brings you syslog-ng related news. FEATURED NEWS syslog-ng at conferences ------------------------ This year, we plan to present syslog-ng in many conferences, or just participate conferences where many syslog-ng users are expected to show up. If you want to meet someone from the syslog-ng team, come to one of our presentations or drop an e-mail so we can find each other. We are also very interested to hear suggestions, where syslog-ng should be presented! Here is a list of confirmed events, which will be updated regularly in later newsletters: FOSDEM, 1-2 February, Brussels, Belgium: https://fosdem.org/2014/, giving a talk titled: “Babelfish for DevOps: syslog-ng” Fedora, JBoss and RedHat developers conference, 7-9 February, Brno, Czech Republic: http://devconf.cz/, Participating only Open Source Data Center Conference, 8-10 April, Berlin, Germany: http://www.netways.de/osdc/, Giving a talk titled "Monitoring with syslog-ng, Riemann and Kibana" Check out syslog-ng 3.6 pre-alpha! ---------------------------------- While syslog-ng 3.6 still did not even reach an alpha release, it received already a lot of development. So, while it might still eat your logs for lunch, those who are interested in where syslog-ng is heading should check out syslog-ng 3.6 from git. Unlike previous syslog-ng versions, it is developed in a unified git repository without a version string attached: https://github.com/balabit/syslog-ng While most changes are under the hood, there are also some user visible features like pseudofile destination: https://github.com/balabit/syslog-ng/commit/be381a4e3f3eca0695f5976678ea26c… Nodejs support was also added tosyslog-ng: use the widespread winston logging API, and syslog-ng will process its JSON formatted messages: https://bazsi.blogs.balabit.com/2014/01/nodejs-support-in-syslog-ng/ And if you look at the stats at http://www.ohloh.net/p/syslog-ng you can see a healthy growth of the code base and in the number of contributors. Thank you for your support! syslog-ng incubator ------------------- The syslog-ng incubator is a collection of tools and modules which are not (yet) part of the official repository. It has some very interesting code in it, like a riemann or an RSS destination, but until now it was completely undocumented. Not any more: http://asylum.madhouse-project.org/blog/2013/12/29/the-incubator/ The Incubator also includes a Lua destination, which makes it possible to write simple destination drivers without a line of C. It is still a work in progress, but is an important step into writing modules in other languages. And to make your life easier packages are available in Debian testing, Ubuntu Trustyand for openSUSE in the 3^rd party repositories ( http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… ). PCI DSS 3.0 Continues to Emphasize the Importance of Log Management ------------------------------------------------------------------- The Payment Card Industry Security Standards Council recently released the Data Security Standard 3.0, three years after the prior version. As one of the most important international data security standards, the latest release was eagerly awaited by IT security practitioners. Clarifications make up the bulk of the changes but the standards council changed most of the 12 major requirements to include modified or additional sub-requirements. With PCI DSS 3.0 the standards council has reiterated that log management is a critical part of security best practices. You can read more about it at https://jluby.blogs.balabit.com/2013/12/09/pci-dss-3-0-continues-to-emphasi… SHORT NEWS: * Testing syslog-ng PE on Windows was never so easy: Windows Quick Start Guide for syslog-ng Premium Edition: http://czanik.blogs.balabit.com/2013/12/syslog-ng-on-windows-quick-start-gu… * TLS vulnerabilities in log traffic: syslog-ng vs. BEAST, CRIME and BREACH:http://gyp.blogs.balabit.com/2013/12/syslog-ng-vs-beast-crime-and-br… * Top 6 SANS Essential Categories of Log Reports 2013: http://czanik.blogs.balabit.com/2013/12/top-6-sans-essential-categories-of-… * Shortening Breach Resolution Times with Distributed Log Management: http://jluby.blogs.balabit.com/2013/12/30/shortening-breach-resolution-time… NEW RELEASES * syslog-ng 3.5.3: https://lists.balabit.hu/pipermail/syslog-ng-announce/2013-December/000177.… * syslog-ng 3.4.7: https://lists.balabit.hu/pipermail/syslog-ng-announce/2013-December/000178.… Your feedback and news tips about the next issue is welcome at documentation(a)balabit.com <mailto:documentation@balabit.com> To read this newsletter on-line, visit: http://insider.blogs.balabit.com/ -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

[PATCH] confgen: backticks weren't processed in generated configuration part
by Juhász Viktor 28 Dec '13

28 Dec '13

confgen: backticks weren't processed in generated configuration part produced by confgen module. The reason of this problem was, that the cfg_lexer_subst_args was called only for read lines from configuration files (YY_INPUT) and for the processed blocks. If process the backticks for a cfg input buffer is a must, the cfg_lexer_subst_args has to be called explicit. Now the confgen module call the it before set the include buffer to the lexer Signed-off-by: Juhász Viktor <jviktor(a)balabit.hu> --- modules/confgen/confgen-plugin.c | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/modules/confgen/confgen-plugin.c b/modules/confgen/confgen-plugin.c index 519a660..9831962 100644 --- a/modules/confgen/confgen-plugin.c +++ b/modules/confgen/confgen-plugin.c @@ -40,6 +40,7 @@ confgen_generate(CfgLexer *lexer, gint type, const gchar *name, CfgArgs *args, g gchar *exec = (gchar *) user_data; gsize res; gchar buf[256]; + gchar *old_value; g_snprintf(buf, sizeof(buf), "%s confgen %s", cfg_lexer_lookup_context_name_by_type(type), name); if (!cfg_args_validate(args, NULL, buf)) @@ -79,6 +80,10 @@ confgen_generate(CfgLexer *lexer, gint type, const gchar *name, CfgArgs *args, g g_free(value); return FALSE; } + + old_value=value; + value = cfg_lexer_subst_args(lexer->globals, NULL, NULL, value, &value_len); + g_free(old_value); if (!cfg_lexer_include_buffer(lexer, buf, value, value_len)) { g_free(value); -- 1.8.3.2

2 2

syslog-ng 3.5.3 has been released
by devel＠balabit.hu 25 Dec '13

25 Dec '13

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.5.3 SUMMARY : new stable release DATE : Dec 24, 2013 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.5.3) has been released. For latest fixes in the 3.5.x feature branch you are recommended to upgrade to this version. CHANGES: 3.5.3 Tue, 24 Dec 2013 19:53:05 +0100 This is the second bug-fix release for the 3.5.x series, with a number of important changes over the previous release. Upgrading to this version is highly recommended. Bugfixes ======== * Node flag propagation has been fixed, so flags(final) works the same as it did in 3.3, when embedded log statements are used too. * Improvements were made to the build system, to make it more reliable when building in parallel. * Using the value-pairs() framework, either via the $(format-json) template function or any of the destinations that make use of it, will now correctly skip and ignore empty values, even in SDATA. * The configure script will correctly detect newer versions of json-c too, without compatibility symlinks being present. * A file descriptor and memory leak has been fixed in the syslog() destination. * The pdbtool program will warn in test mode when a test message has no program attribute. [BugZilla #263] * If the PID property is overwritten, syslog-ng will now stop using the cached MSGHDR value, and therefore, use the newly set PID instead, as it was supposed to. * The localport() option of various network destinations has been restored, and will work correctly now. [BugZilla #266] * The upstart service file in contrib has been updated to use a different process mode (background), as we do not need a supervisor when running under upstart. This also means that upstart will be far less confused. * Manual pages are now always included in the tarball. Credits ======= syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Andras Mitzki <micek(a)balabit.hu> Balazs Scheidler <bazsi(a)balabit.hu> Chris Johnson <chris.johnson3(a)hp.com> Fabien Wernli <cpan(a)faxm0dem.org> Gergely Nagy <algernon(a)balabit.hu> Lucas McLane <lucas(a)clicksecurity.com> Peter Czanik <czanik(a)balabit.hu> Sebastiaan Hoogeveen <zebaz(a)xs4all.nl> Tobias Schwab <tobias.schwab(a)dynport.de> DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

3 2

syslog-ng 3.4.7, 3.5.3 and incubator packages for openSUSE / SLES
by Peter Czanik 25 Dec '13

25 Dec '13

Hello, Here is my Christmas present for you: openSUSE packages for syslog-ng updated and syslog-ng incubator package added. As it's Christmas, it received only very limited testing (only openSUSE 13.1 x86_65) and incubator lacks riemann support (that needs packaging an additional support library). 3.5 man pages are still a kind of broken, so SLES support is missing there. Packages are available at: https://build.opensuse.org/project/show/home:czanik:syslog-ng34 https://build.opensuse.org/project/show/home:czanik:syslog-ng35 Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/ https://twitter.com/PCzanik

1 0

syslog-ng 3.4.7 has been released
by devel＠balabit.hu 24 Dec '13

24 Dec '13

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.4.7 SUMMARY : new stable release DATE : Dec 24, 2013 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.4.7) has been released. For latest fixes in the 3.4.x feature branch you are recommended to upgrade to this version. CHANGES: 3.4.7 Tue, 24 Dec 2013 21:07:34 +0100 This is the sixth bug-fix release for the 3.4.x series, with a number of important fixes over the 3.4.6 release. Upgrading to this release is highly recommended. Bugfixes ======== * Node flag propagation has been fixed, so flags(final) works the same as it did in 3.3, when embedded log statements are used too. * Using the value-pairs() framework, either via the $(format-json) template function or any of the destinations that make use of it, will now correctly skip and ignore empty values, even in SDATA. * A file descriptor and memory leak has been fixed in the syslog() destination. * The pdbtool program will warn in test mode when a test message has no program attribute. [BugZilla #263] * If the PID property is overwritten, syslog-ng will now stop using the cached MSGHDR value, and therefore, use the newly set PID instead, as it was supposed to. * The localport() option of various network destinations has been restored, and will work correctly now. [BugZilla #266] * The upstart service file in contrib has been updated to use a different process mode (background), as we do not need a supervisor when running under upstart. This also means that upstart will be far less confused. Credits ======= syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Andras Mitzki <micek(a)balabit.hu> Balazs Scheidler <bazsi(a)balabit.hu> Chris Johnson <chris.johnson3(a)hp.com> Fabien Wernli <cpan(a)faxm0dem.org> Gergely Nagy <algernon(a)balabit.hu> Lucas McLane <lucas(a)clicksecurity.com> Peter Czanik <czanik(a)balabit.hu> Sebastiaan Hoogeveen <zebaz(a)xs4all.nl> Tobias Schwab <tobias.schwab(a)dynport.de> DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

[Bug 265] New: syslog-ng does not output the $PID of the process, even when forced
by bugzilla＠bugzilla.balabit.com 23 Dec '13

23 Dec '13

https://bugzilla.balabit.com/show_bug.cgi?id=265 Summary: syslog-ng does not output the $PID of the process, even when forced Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: support(a)sikkerhed.org Type of the Report: --- Estimated Hours: 0.0 I'm researching syslog-ng, in order to set it up on all my machines, and have them log to a central server on the network. Things are going great, except I can't seem to force syslog-ng to report the $PID of the program that logs, so sometimes I get lines that have missing $PIDS: (anonymized, but based on real log line) 2013-12-05T19:40:44.388099+01:00 web15.example.org pop3d: Connection, ip=[::ffff:x.y.z.w] Whereas I was able to change pureftpd to log with pid: 2013-12-05T19:40:44.639127+01:00 web15.example.org pure-ftpd[872804]: (?(a)x.y.z.w) [INFO] New connection from x.y.z.w Unfortunately, courier-imap does not have such a logging option. I thought that using $PID in the output template would force syslog-ng to log the PID of the reporting process, but this does not seem to be the case. Is it really true that there is no way to force syslog-ng to report the PID? I know that rsyslog has this feature, but I would much rather use syslog-ng :) -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

3 6

[Bug 186] New: resettable stats
by bugzilla＠bugzilla.balabit.com 21 Dec '13

21 Dec '13

https://bugzilla.balabit.com/show_bug.cgi?id=186 Summary: resettable stats Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: szaki(a)balabit.hu Type of the Report: --- Estimated Hours: 0.0 According to https://lists.balabit.hu/pipermail/syslog-ng/2011-October/017457.html It would be a good idea to be able to reset the statistic, when needed, without full restart, or any magic like altering the config a bit and reload. So I suggest the enable_stats_reset(yes) default option, for sources and destinations (everywhere are statcounters defined), and the syslog-ng-ctl reset-stats command - to reset the statistic where the enable_stats_reset(yes) option allows this. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

[Bug 262] New: patterndb: Computed <values> are not expanded in context-id
by bugzilla＠bugzilla.balabit.com 20 Dec '13

20 Dec '13

https://bugzilla.balabit.com/show_bug.cgi?id=262 Summary: patterndb: Computed <values> are not expanded in context-id Product: syslog-ng Version: 3.5.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.balabit(a)faxm0dem.org Type of the Report: --- Estimated Hours: 0.0 Created an attachment (id=87) --> (https://bugzilla.balabit.com/attachment.cgi?id=87) patterndb ruleset It seems the values are not being expanded as expected in the context-id: -- pdbtool test /tmp/test.pdb -vd Module loaded and initialized successfully; module='syslogformat' Module loaded and initialized successfully; module='basicfuncs' Testing message program='' message='the big "brown" fox' patterndb rule matches; rule_id='A' Advancing patterndb current time because of an incoming message; utc='-1' Correllation context lookup failure, starting a new context; rule='A', context='brown.', context_timeout='0', context_expiration='-1' Match name='.classifier.rule_id', value='A', expected='A' Match name='matched.var', value='brown', expected='brown' -- as you can see, context-id='brown.' instead of 'brown.brown' as expected by expanding '${matched.var}.${computed.var}' -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 5

[Bug 263] New: patterndb: missing "program" attribute in test_message silently ignored
by bugzilla＠bugzilla.balabit.com 20 Dec '13

20 Dec '13

https://bugzilla.balabit.com/show_bug.cgi?id=263 Summary: patterndb: missing "program" attribute in test_message silently ignored Product: syslog-ng Version: 3.5.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: bugzilla.balabit(a)faxm0dem.org Type of the Report: --- Estimated Hours: 0.0 Although the attached pdb validates, the test messages are not run, because "program" attribute is missing. There should be a warning, or something: -- pdbtool --validate test /tmp/test2.pdb -dv Module loaded and initialized successfully; module='syslogformat' Module loaded and initialized successfully; module='basicfuncs' /tmp/test2.pdb validates -- -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

[Bug 266] New: localport option does not bind to a specific port
by bugzilla＠bugzilla.balabit.com 20 Dec '13

20 Dec '13

https://bugzilla.balabit.com/show_bug.cgi?id=266 Summary: localport option does not bind to a specific port Product: syslog-ng Version: unspecified Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: zebaz(a)xs4all.nl Type of the Report: bug Estimated Hours: 0.0 The localport option to specify a source port when opening a TCP connection no longer seems to be working. For testing I used the following simplified configurations: @version: 3.5 source src { unix-stream("/dev/log" max-connections(256)); internal(); file("/proc/kmsg"); }; destination logcentral { tcp("logcentral" port(514) localport(1014)); }; log { source(src); destination(logcentral); }; When running syslog-ng -d the relevant output is: Syslog connection established; fd='8', server='AF_INET(<ip removed>:514)', local='AF_INET(0.0.0.0:0)' I tried some variations in the configuration: - source port > 1024 gives the same result - specifying the localip option binds the local socket explicitly to that IP address but still not to a specific port Behaviour has been checked to occur with versions 3.4.2 and 3.4.6 from Gentoo as well as 3.5.2 from the upstream site (compiled from source). Other installations with identical kernel versions run older syslog-ng installations (3.2.x and 3.3.x) with the same settings without a problem. -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 11