October 2013 - syslog-ng - lists.balabit.hu

[Bug 255] New: segmentation faults often
by bugzilla＠bugzilla.balabit.com 02 Nov '13

02 Nov '13

https://bugzilla.balabit.com/show_bug.cgi?id=255 Summary: segmentation faults often Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: blocker Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: erempel(a)uvic.ca Type of the Report: bug Estimated Hours: 0.0 In my setup, which is big, using syslog-ng 3.4.4 I get segmentation faults 2-3 times per minute. 3.4.1 did this once every couple of weeks, and 3.4.2 was worse, but 3.4.4 is completely unusable for me. What would you like from me to aid tracking this down? -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

3 15

patterndb + format_json 100% CPU
by Alexandre Biancalana 27 Oct '13

27 Oct '13

Hi list, Running syslog-ng with self written patterndb and format_json output the process is hitting 100% of CPU looks like it's freezed (infinite loop ?) and nothing is written to the output. Follow the link to syslog-ng.conf, patterndb file and example logfile: syslog-ng.conf - http://pastebin.com/QD6huzA3 httpd.xml (patterndb) - http://pastebin.com/kF9zJHYH all.log - http://pastebin.com/syusGF2H Was possible to reproduce that with 3.4.4 and 3.5.0beta3. Regards, Alexandre

3 10

Question about postfix, multiline and tls
by Clayton Dukes 26 Oct '13

26 Oct '13

Hi Folks, I'm trying to get syslog-ng to send postfix events to a server as a single event, instead of multiple events for the same log message. I've posted the question on stack exchange and thought I was see if anyone here may know the answer: http://stackoverflow.com/questions/19609142/syslog-ng-support-for-postfix-m… ______________________________________________________________ Clayton Dukes ______________________________________________________________

1 0

3.5rc1 packages for openSUSE / SLES
by Peter Czanik 24 Oct '13

24 Oct '13

Hello, If you want to test 3.5rc1 on openSUSE / SLES but afraid to compile it yourself, my unofficial packages are available at https://build.opensuse.org/package/show/home:czanik:syslog-ng35/syslog-ng It received some testing on openSUSE 12.3, but available for a lot other platforms. Bye, -- Peter Czanik (CzP) <czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

1 0

syslog-ng 3.5.0rc1 has been released
by devel＠balabit.hu 24 Oct '13

24 Oct '13

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.5.0rc1 SUMMARY : new release candidate release DATE : Oct 24, 2013 ------------------------------------------------------------------------------ DESCRIPTION: A new release candidate of syslog-ng Open Source Edition (3.5.0rc1) has been released. For latest fixes in the 3.5.x feature branch you are recommended to upgrade to this version. CHANGES: 3.5.0rc1 Thu, 24 Oct 2013 10:48:51 +0200 This is the first Release Candidate of the syslog-ng 3.5 development branch, with stability and reliability improvements. Being a release candidate, this release should be very close to production ready. Bugfixes ======== * The program() source and destination now properly terminates child processes on reload and shutdown. * The AMQP, SMTP, MongoDB, Redis and STOMP destinations now work correctly across reloads. * A rare race condition in the log message reference cache was corrected, fixing a crash. * The JSON parser now understands null-typed values, and ignores the whole key silently when it encounters one. * Using the subst() rewrite rule when the same source is present in multiple logpaths does not crash anymore. Miscellaneous ============= * The autogen.sh script is included in the distributed tarball. Known Bugs ========== * The afstreams module is broken, does not compile, and does not work. This will be corrected before the final 3.5.1 release. Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Balazs Scheidler <bazsi(a)balabit.hu> Evan Rempel <erempel(a)uvic.ca> Gergely Nagy <algernon(a)balabit.hu> Jose Pedro Oliveira <jpo(a)di.uminho.pt> Michal Privoznik <miso.privoznik(a)gmail.com> Nicolas Szalay <nico(a)rottenbytes.info> Ryan Frederick DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

[Bug 256] New: rewrite + subst make syslog-ng crash
by bugzilla＠bugzilla.balabit.com 24 Oct '13

24 Oct '13

https://bugzilla.balabit.com/show_bug.cgi?id=256 Summary: rewrite + subst make syslog-ng crash Product: syslog-ng Version: 3.4.x Platform: PC OS/Version: Linux Status: NEW Severity: normal Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: nico(a)rottenbytes.info Type of the Report: bug Estimated Hours: 0.0 Hi, I've been using syslog-ng 3.4.4 (from madhouse-project debian packages) and I'm getting crashes of the daemon on rewrite rules. here is the scheme : rewrite r_services { subst("postfix/.*", "postfix", value("PROGRAM"), flags("global")); subst("nagios.*", "nagios", value("PROGRAM"), flags("global")); subst("jftpgw", "ftp-proxy", value("PROGRAM"), flags("global")); subst("ftp-child", "ftp-proxy", value("PROGRAM"), flags("global")); }; log { source(s_local); source(s_remote); rewrite(r_services); filter(f_services); destination(df_services); flags(final); }; will make the daemon crash. rewrite + set rules are NOT impacted and work fine. Using rewrite + set and a condition will work too for example : rewrite r_services { set("postfix", value("PROGRAM") condition(program("postfix/.*"))); }; works. When the program crashes I get the following error : ERROR:../../../lib/logmsg.c:535:log_msg_set_value_indirect: assertion failed: (!log_msg_is_write_protected(self)) Feel free to contact me if you need more details. Regards -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 6

syslog-ng OSE supported versions, EoL times, etc
by Gergely Nagy 22 Oct '13

22 Oct '13

Hi! We've been thinking about how to approach the problem of supporting syslog-ng OSE versions, the end-of-life of them, and related issues. I've been maintaining syslog-ng branches for about a year and a half now, and the time and effort required to do that is only becoming clear now that I have a reasonable sample size. There was no clear communication about how long each version of syslog-ng OSE would be supported, and when they will reach their end of life, which made it harder for distributions and pretty much everyone else to plan ahead and schedule time for migrating to a new version. We intend to rectify that now, and my plan is outlined below: We would - at any one time - maintain four branches of syslog-ng (including the bleeding edge development branch). The other three would be: a critical-bug-fix-only maintenance release (this is currently 3.3), a recommended stable release (3.4), and the upcoming feature release (3.5beta at the moment). Whenever the development branch gets a beta release, it would automatically become the upcoming feature release, the former feature release would get promoted to stable, and the former stable becomes the maintenance branch. In this scheme, 3.3 would be maintained until 3.6.0beta1, at which point, it would be dropped from the supported set, and 3.4 would take its place, and 3.5 would become the recommended stable release. Of course, it's still hard to plan ahead without dates, but dates are something I cannot give at this point. The best I can do is send an announcement once 3.6 hits the alpha stage, and give a rough estimate on when it would hit beta. That's a few months in advance, and by that time, 3.4 will be out long enough for folk to migrate over, thus 3.3 shouldn't be in much use anymore. With the current pace of development, this means that a syslog-ng OSE release will be maintained for about 2-2.5 years, and I expect it will not go down by much more than half a year, meaning 1.5-2 years of maintenance. And to make it clear what each branch means, let me expand on that a bit: * Maintenance: The former stable branch, which receives critical bug fixes only. People are strongly encouraged to migrate from this to the stable one, but the branch is still supported to some extent, to make it easier for distributions, among others. Critical bugs are security issues, serious regressions, and so on - I reserve the right to decide which bugs to consider critical. Fixes from any other branch are not backported, unless they're critical. Currently the 3.3 branch fills this role. * Stable: The stable branch, receiving all kinds of bug fixes, including smaller fixes backported from newer branches (including the next feature release and the development branch too). It may also receive very small features too, which do not affect the package as a whole: things like the $(format-json) improvement to allow comma-separated scopes, or being able to specify a template for the JSON parser. Currently the 3.4 branch fills this role. * Feature: The next stable branch, which we consider stable enough, but has not received as wide testing as the stable branch yet. This gets the most attention, as the idea is to stabilise it as soon as possible. Other than being newer than the Stable branch, this is treated in exactly the same way. Currently the 3.5 branch fills this role. * Development: The branch where all development happens. While we try to keep it stable, regressions and breakage is more likely here, and new features are not guaranteed to remain the same during the development cycle, so building production on this branch is not advised. New features are developed against this branch, and the Incubator's[1] master branch follows the development branch too. The 3.6 branch will fill this role, once opened. As for the Incubator[1]: it will only have two branches: one for the latest feature release, and another for development. Maintenance and Stable versions of syslog-ng will not be supported in there. [1]: https://github.com/algernon/syslog-ng-incubator The Incubator is a place for experimental modules that are either not ready to be merged into syslog-ng proper, or are kept separate for any other reason. -- |8]

1 0

mongodb stats crash
by Balazs Scheidler 20 Oct '13

20 Oct '13

Hi, I have encountered a fixed crash in 3.5 master in the mongodb() driver, it aborts in stats_unregister_counter() it seems that afmongodb_format_stats_instance() returns different values in init() and deinit(). This is my config: mongodb(value-pairs(pair('foo', "$STAMP")) frac-digits(3)); This is the backtrace: [Thread 0x7ffff7fca700 (LWP 27493) exited] ** ERROR:/home/bazsi/zwa/work/syslog-ng-ose-3.5/syslog-ng-ose/lib/stats.c:302:stats_unregister_counter: assertion failed: (sc && (sc->live_mask & (1 << type)) && &sc->counters[type] == (*counter)) Program received signal SIGABRT, Aborted. 0x00007ffff7275037 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory. (gdb) bt #0 0x00007ffff7275037 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56 #1 0x00007ffff7278698 in __GI_abort () at abort.c:90 #2 0x00007ffff788e3b6 in g_assertion_message () from /lib/x86_64-linux-gnu/libglib-2.0.so.0 #3 0x00007ffff788e914 in g_assertion_message_expr () from /lib/x86_64-linux-gnu/libglib-2.0.so.0 #4 0x00007ffff7b69ef8 in stats_unregister_counter (source=533, id=0x61cd10 "d_local#0", instance=0x7ffff4d26c40 <persist_name.15015> "mongodb,127.0.0.1,27017,syslog,messages", type=SC_TYPE_STORED, counter=0x6e4cd0) at /home/bazsi/zwa/work/syslog-ng-ose-3.5/syslog-ng-ose/lib/stats.c:302 #5 0x00007ffff7b5b838 in log_threaded_dest_driver_deinit_method (s=0x6e4c10) at /home/bazsi/zwa/work/syslog-ng-ose-3.5/syslog-ng-ose/lib/logthrdestdrv.c:162 #6 0x00007ffff7b49014 in log_pipe_deinit (s=0x6e4c10) at /home/bazsi/zwa/work/syslog-ng-ose-3.5/syslog-ng-ose/lib/logpipe.h:268 #7 0x00007ffff7b4afad in cfg_tree_stop (self=0x613f30) at /home/bazsi/zwa/work/syslog-ng-ose-3.5/syslog-ng-ose/lib/cfg-tree.c:1080 #8 0x00007ffff7b44908 in cfg_deinit (cfg=0x613dc0) at /home/bazsi/zwa/work/syslog-ng-ose-3.5/syslog-ng-ose/lib/cfg.c:225 #9 0x00007ffff7b607b1 in main_loop_exit_finish () at /home/bazsi/zwa/work/syslog-ng-ose-3.5/syslog-ng-ose/lib/mainloop.c:646 #10 0x00007ffff7b60407 in main_loop_io_worker_sync_call (func=0x7ffff7b60799 <main_loop_exit_finish>) at /home/bazsi/zwa/work/syslog-ng-ose-3.5/syslog-ng-ose/lib/mainloop.c:470 #11 0x00007ffff7b607d0 in main_loop_exit_timer_elapsed (arg=0x0) at /home/bazsi/zwa/work/syslog-ng-ose-3.5/syslog-ng-ose/lib/mainloop.c:653 #12 0x00007ffff7b96568 in iv_run_timers () from /home/bazsi/zwa/install/syslog-ng-ose-3.5/lib/libsyslog-ng-3.5.0beta3.so #13 0x00007ffff7b99230 in iv_main () from /home/bazsi/zwa/install/syslog-ng-ose-3.5/lib/libsyslog-ng-3.5.0beta3.so #14 0x00007ffff7b60ba7 in main_loop_run () at /home/bazsi/zwa/work/syslog-ng-ose-3.5/syslog-ng-ose/lib/mainloop.c:796 #15 0x0000000000401a67 in main (argc=1, argv=0x7fffffffd648) at /home/bazsi/zwa/work/syslog-ng-ose-3.5/syslog-ng-ose/syslog-ng/main.c:269 (gdb) Stepping through format_stats_instance() at init time: Breakpoint 3, afmongodb_dd_format_stats_instance (d=0x6e4c10) at /home/bazsi/zwa/work/syslog-ng-ose-3.5/syslog-ng-ose/modules/afmongodb/afmongodb.c:189 189 MongoDBDestDriver *self = (MongoDBDestDriver *)d; (gdb) n 192 if (self->port == MONGO_CONN_LOCAL) (gdb) p self->port $1 = 0 (gdb) n 197 "mongodb,%s,%u,%s,%s", self->address, self->port, self->db, self->coll); (gdb) 196 g_snprintf(persist_name, sizeof(persist_name), (gdb) l 191 192 if (self->port == MONGO_CONN_LOCAL) 193 g_snprintf(persist_name, sizeof(persist_name), 194 "mongodb,%s,%s,%s", self->address, self->db, self->coll); 195 else 196 g_snprintf(persist_name, sizeof(persist_name), 197 "mongodb,%s,%u,%s,%s", self->address, self->port, self->db, self->coll); 198 return persist_name; 199 } 200 (gdb) n 197 "mongodb,%s,%u,%s,%s", self->address, self->port, self->db, self->coll); (gdb) 196 g_snprintf(persist_name, sizeof(persist_name), (gdb) 197 "mongodb,%s,%u,%s,%s", self->address, self->port, self->db, self->coll); (gdb) 196 g_snprintf(persist_name, sizeof(persist_name), (gdb) 197 "mongodb,%s,%u,%s,%s", self->address, self->port, self->db, self->coll); (gdb) 196 g_snprintf(persist_name, sizeof(persist_name), (gdb) 198 return persist_name; (gdb) p persist_name $2 = "mongodb,(null),0,syslog,messages", '\000' <repeats 991 times> At deinit time: Breakpoint 3, afmongodb_dd_format_stats_instance (d=0x6e4c10) at /home/bazsi/zwa/work/syslog-ng-ose-3.5/syslog-ng-ose/modules/afmongodb/afmongodb.c:189 189 MongoDBDestDriver *self = (MongoDBDestDriver *)d; (gdb) n 192 if (self->port == MONGO_CONN_LOCAL) (gdb) p self->port $3 = 27017 (gdb) p self->address $4 = (gchar *) 0x61d290 "127.0.0.1" (gdb) n 197 "mongodb,%s,%u,%s,%s", self->address, self->port, self->db, self->coll); (gdb) 196 g_snprintf(persist_name, sizeof(persist_name), (gdb) 197 "mongodb,%s,%u,%s,%s", self->address, self->port, self->db, self->coll); (gdb) 196 g_snprintf(persist_name, sizeof(persist_name), (gdb) 197 "mongodb,%s,%u,%s,%s", self->address, self->port, self->db, self->coll); (gdb) 196 g_snprintf(persist_name, sizeof(persist_name), (gdb) 197 "mongodb,%s,%u,%s,%s", self->address, self->port, self->db, self->coll); (gdb) 196 g_snprintf(persist_name, sizeof(persist_name), (gdb) 198 return persist_name; (gdb) p persist_name $5 = "mongodb,127.0.0.1,27017,syslog,messages", '\000' <repeats 984 times> This patch solves for me, however I'm not 100% certain it's the proper fix and the other LogThreadedDestDriver instances might be affected too. diff --git a/modules/afmongodb/afmongodb.c b/modules/afmongodb/afmongodb.c index 6504a63..b625c2c 100644 --- a/modules/afmongodb/afmongodb.c +++ b/modules/afmongodb/afmongodb.c @@ -516,8 +516,6 @@ afmongodb_dd_init(LogPipe *s) GlobalConfig *cfg = log_pipe_get_config(s); ValuePairsTransformSet *vpts; - if (!log_threaded_dest_driver_init_method(s)) - return FALSE; log_template_options_init(&self->template_options, cfg); @@ -566,6 +564,9 @@ afmongodb_dd_init(LogPipe *s) evt_tag_str("collection", self->coll), NULL); + if (!log_threaded_dest_driver_init_method(s)) + return FALSE; + log_threaded_dest_driver_start(&self->super); return TRUE; In case we're moving this to this point, I'd say that the threaded_dest_driver_start() could be moved into the log_threaded_dest_driver_init_method() function, can't it? Can you have a look? Thanks.

2 6

syslog-ng 3.4.4 has been released
by devel＠balabit.hu 18 Oct '13

18 Oct '13

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.4.4 SUMMARY : new stable release DATE : Oct 17, 2013 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.4.4) has been released. For latest fixes in the 3.4.x feature branch you are recommended to upgrade to this version. CHANGES: 3.4.4 Thu, 17 Oct 2013 12:16:40 +0200 This is the third bug-fix release for the 3.4.x series, with important fixes over the 3.4.3 release. Upgrading to this release is highly recommended. Bugfixes ======== * Fixed the log message reference counting in the AMQP destination, thus stopping it from crashing. * Fixed a possible crash in the AMQP destination that could be triggered when the AMQP server was unreachable at the time of a syslog-ng reload or shutdown. * pdbtool no longer segfaults when used with the --module switch [#249] * pdbtool test exits with a return value of 2 if the test value matched a different pattern than expected. [#241] * The inherit-properties attribute was added to the patterndb XSD. [#251] * A memory corruption was fixed in the @STRING@ parser of db-parser(). [#254] * Corrected a mistake in the afsocket module that lead to crashes on reload when unix-stream or unix-dgram destinations were in use. * Excludes now work properly with value-pairs(), and they do not get ignored if the value to exclude is in the default set. * The program() source and destination now correctly kill the started child on reload and shutdown. * The file destination does not use lseek() anymore. * When setting default template options, all random memory junk is cleared from the object first. * The order of header include paths was fixed in the test suite, so it should compile cleanly even in the presence of potentially conflicting headers. Features ======== * The $(format-json) template function, and anything else that may use the same value-pairs command line parser now allow specifying multiple scopes separated by commas (as opposed to having to use the --scope switch multiple times) [#248] Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Alexandre Biancalana <biancalana(a)gmail.com> Balazs Scheidler <bazsi(a)balabit.hu> Balint Kovacs <blint(a)balabit.hu> Evan Rempel <erempel(a)uvic.ca> Fabien Wernli <cpan(a)faxm0dem.org> Gergely Nagy <algernon(a)balabit.hu> Lucas McLane <lucas(a)clicksecurity.com> Marc Falzon <marc.falzon(a)cloudwatt.com> Michal Privoznik <miso.privoznik(a)gmail.com> Ryan Frederick Valentijn Sessink <valentijn(a)sessink.nl> DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

5 6

ANN: Unofficial Debian/Ubuntu repo switching to 3.4 by default
by Gergely Nagy 17 Oct '13

17 Oct '13

As announced before[1], the syslog-ng Debian/Ubuntu repository[2] I provide will switch to syslog-ng 3.4 as default. This means that if you're using source lines like below, you'll get the 3.4.4 update coming later today. deb http://packages.madhouse-project.org/debian wheezy syslog-ng If you want to remain with 3.3, use the syslog-ng-3.3 component. The 3.3 branch will also be correctly labelled as "maintained", and 3.4 gets promoted to "stable". 3.5 packages will start coming with 3.5.0rc1 due next week. [1]: https://lists.balabit.hu/pipermail/syslog-ng/2013-March/020158.html [2]: http://asylum.madhouse-project.org/projects/debian/ -- |8]

1 0