- syslog-ng - lists.balabit.hu

3.4 on opensuse factory
by Peter Czanik 01 May '12

01 May '12

Hello, When trying to run syslog-ng 3.4 on openSUSE factory, the supervisor process starts, but it's child segfaults continuously. I tried to revert the big nvtable patch, which solved my problem previously, but it did not help. Here are two backtraces, practically the same: With the nvtable patch: (gdb) bt #0 0x00007f3e447516c6 in __memcpy_ssse3_back () from /lib64/libc.so.6 #1 0x00007f3e44f1a3ff in nv_table_add_value () from /lib64/libsyslog-ng-3.4.0alpha1.so #2 0x00007f3e44f0858f in log_msg_set_value () from /lib64/libsyslog-ng-3.4.0alpha1.so #3 0x00007f3e44f015db in log_src_driver_queue_method () from /lib64/libsyslog-ng-3.4.0alpha1.so #4 0x00007f3e44f12bfe in log_source_queue () from /lib64/libsyslog-ng-3.4.0alpha1.so #5 0x00007f3e44efa008 in afinter_source_post () from /lib64/libsyslog-ng-3.4.0alpha1.so #6 0x00007f3e44f338d4 in iv_event_run_pending_events () from /lib64/libsyslog-ng-3.4.0alpha1.so #7 0x00007f3e44f33c49 in iv_event_raw_got_event () from /lib64/libsyslog-ng-3.4.0alpha1.so #8 0x00007f3e44f3534a in iv_main () from /lib64/libsyslog-ng-3.4.0alpha1.so #9 0x00007f3e44f18532 in main_loop_run () from /lib64/libsyslog-ng-3.4.0alpha1.so #10 0x00000000004016e1 in main () With the nvtable patch (https://github.com/bazsi/syslog-ng-3.4/commit/1c3a37764d98d5a37f2cd3c79f7f4…) reverted: (gdb) bt #0 0x00007f6fdb8b86c6 in __memcpy_ssse3_back () from /lib64/libc.so.6 #1 0x00007f6fdc0818bf in nv_table_add_value () from /lib64/libsyslog-ng-3.4.0alpha1.so #2 0x00007f6fdc06f620 in log_msg_set_value () from /lib64/libsyslog-ng-3.4.0alpha1.so #3 0x00007f6fdc0685cb in log_src_driver_queue_method () from /lib64/libsyslog-ng-3.4.0alpha1.so #4 0x00007f6fdc079dbe in log_source_queue () from /lib64/libsyslog-ng-3.4.0alpha1.so #5 0x00007f6fdc060ff8 in afinter_source_post () from /lib64/libsyslog-ng-3.4.0alpha1.so #6 0x00007f6fdc09afc4 in iv_event_run_pending_events () from /lib64/libsyslog-ng-3.4.0alpha1.so #7 0x00007f6fdc09b339 in iv_event_raw_got_event () from /lib64/libsyslog-ng-3.4.0alpha1.so #8 0x00007f6fdc09ca3a in iv_main () from /lib64/libsyslog-ng-3.4.0alpha1.so #9 0x00007f6fdc07f942 in main_loop_run () from /lib64/libsyslog-ng-3.4.0alpha1.so #10 0x00000000004016e1 in main () Is it something in syslog-ng, or I should rather report it at openSUSE? Bye, -- Peter Czanik (CzP)<czanik(a)balabit.hu> BalaBit IT Security / syslog-ng upstream http://czanik.blogs.balabit.com/

3 3

[PATCH] contrib/systemd: Alias the service to syslog.service
by Gergely Nagy 01 May '12

01 May '12

In order to be considered a syslog service, we need to Alias ourselves to syslog.service. Doing this will make systemd consider syslog-ng a valid syslogd service, without the alias, one has to do something similar manually. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- contrib/systemd/syslog-ng.service | 1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/contrib/systemd/syslog-ng.service b/contrib/systemd/syslog-ng.service index 85c91e5..f9b9fe2 100644 --- a/contrib/systemd/syslog-ng.service +++ b/contrib/systemd/syslog-ng.service @@ -10,3 +10,4 @@ StandardOutput=null [Install] WantedBy=multi-user.target +Alias=syslog.service -- 1.7.9.1

2 1

File permission in syslog-ng
by Ankit Mahawar 01 May '12

01 May '12

Hi All, If the file permission is not specified in syslog-ng.conf file then it uses the file permission as default which is 600 . I noticed it and want to confirm this behavior of syslog-ng . Also if we define the permission as perm(644) in options tag of syslog-ng.conf then does that permission is set for all the files ? Please provide your inputs as I am understanding this utility. Thanks Ankit =====-----=====-----===== Notice: The information contained in this e-mail message and/or attachments to it may contain confidential or privileged information. If you are not the intended recipient, any dissemination, use, review, distribution, printing or copying of the information contained in this e-mail message and/or attachments to it are strictly prohibited. If you have received this communication in error, please notify us by reply e-mail or telephone and immediately and permanently delete the message and any attachments. Thank you

2 1

Antw: [PATCH] afinter: Make the parent class LogSrcDriver, instead of LogDriver.
by Peter Eisenlohr 01 May '12

01 May '12

Hi, since the large NVTable patch has been re-applied (1c3a37764d), this patch should be committed soon, as syslog-ng currently still crashes when a mark is logged. Peter Dipl.-Inf. Peter Eisenlohr Geschäftsbereich Logistik INFORM GmbH, Pascalstr. 23, 52076 Aachen, Germany Telefon: (+49) 2408 9456-6000 E-Mail: peter.eisenlohr(a)inform-software.com, Web: http://www.inform-software.com INFORM Institut fur Operations Research und Management GmbH Registered AmtsG Aachen HRB1144 Gfhr. Adrian Weiler >>> Gergely Nagy 06.04.12 11.36 Uhr >>> The parent of AFInterSourceDriver should be LogSrcDriver, because that's the struct we pass down to it anyway, and other parts of the code will cast it back to LogSrcDriver, too. But if its parent is LogDriver, as it was, we were corrupting the later parts of the original LogSrcDriver, with values from AFInterSourceDriver. Many thanks to Peter Eisenlohr for initial debugging, and for pinpointing mark messages as the triggers. Reported-by: Peter Czanik Signed-off-by: Gergely Nagy --- lib/afinter.c | 10 +++++----- lib/afinter.h | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/lib/afinter.c b/lib/afinter.c index 23b1c82..b9d0ddf 100644 --- a/lib/afinter.c +++ b/lib/afinter.c @@ -304,7 +304,7 @@ afinter_source_new(AFInterSourceDriver *owner, LogSourceOptions *options) AFInterSource *self = g_new0(AFInterSource, 1); log_source_init_instance(&self->super); - log_source_set_options(&self->super, options, 0, SCS_INTERNAL, owner->super.id, NULL, FALSE); + log_source_set_options(&self->super.super, options, 0, SCS_INTERNAL, owner->super.super.id, NULL, FALSE); afinter_source_init_watches(self); self->super.super.init = afinter_source_init; self->super.super.deinit = afinter_source_deinit; @@ -328,7 +328,7 @@ afinter_sd_init(LogPipe *s) return FALSE; } - log_source_options_init(&self->source_options, cfg, self->super.group); + log_source_options_init(&self->source_options, cfg, self->super.super.group); self->source = afinter_source_new(self, &self->source_options); log_pipe_append(&self->source->super, s); log_pipe_init(&self->source->super, cfg); @@ -370,9 +370,9 @@ afinter_sd_new(void) AFInterSourceDriver *self = g_new0(AFInterSourceDriver, 1); log_src_driver_init_instance((LogSrcDriver *)&self->super); - self->super.super.init = afinter_sd_init; - self->super.super.deinit = afinter_sd_deinit; - self->super.super.free_fn = afinter_sd_free; + self->super.super.super.init = afinter_sd_init; + self->super.super.super.deinit = afinter_sd_deinit; + self->super.super.super.free_fn = afinter_sd_free; log_source_options_defaults(&self->source_options); return (LogDriver *)&self->super.super; } diff --git a/lib/afinter.h b/lib/afinter.h index 108475f..1cd9d78 100644 --- a/lib/afinter.h +++ b/lib/afinter.h @@ -33,7 +33,7 @@ */ typedef struct _AFInterSourceDriver { - LogDriver super; + LogSrcDriver super; LogSource *source; LogSourceOptions source_options; } AFInterSourceDriver; -- 1.7.9.1 ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq

2 1

[PATCH (3.4)] configure.in: Add --disable-smtp support.
by Gergely Nagy 01 May '12

01 May '12

Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- configure.in | 4 ++++ 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/configure.in b/configure.in index f21e130..2d56bbd 100644 --- a/configure.in +++ b/configure.in @@ -178,6 +178,10 @@ AC_ARG_ENABLE(json, [ --disable-json Disable JSON support (default: auto)], [case "${enableval}" in yes) with_json="auto";; *) with_json="${enableval}";; esac],with_json="auto") +AC_ARG_ENABLE(smtp, + [ --disable-smtp Disable SMTP support (default: auto)], + ,,enable_smtp="auto") + AC_ARG_WITH(libesmtp, AC_HELP_STRING([--with-libesmtp=DIR], [use libesmtp library from (prefix) directory DIR]),,) -- 1.7.9.1

2 1

[PATCH] afinter: Make the parent class LogSrcDriver, instead of LogDriver.
by Gergely Nagy 01 May '12

01 May '12

The parent of AFInterSourceDriver should be LogSrcDriver, because that's the struct we pass down to it anyway, and other parts of the code will cast it back to LogSrcDriver, too. But if its parent is LogDriver, as it was, we were corrupting the later parts of the original LogSrcDriver, with values from AFInterSourceDriver. Many thanks to Peter Eisenlohr <peter.eisenlohr(a)inform-software.com> for initial debugging, and for pinpointing mark messages as the triggers. Reported-by: Peter Czanik <czanik(a)balabit.hu> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/afinter.c | 10 +++++----- lib/afinter.h | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/lib/afinter.c b/lib/afinter.c index 23b1c82..b9d0ddf 100644 --- a/lib/afinter.c +++ b/lib/afinter.c @@ -304,7 +304,7 @@ afinter_source_new(AFInterSourceDriver *owner, LogSourceOptions *options) AFInterSource *self = g_new0(AFInterSource, 1); log_source_init_instance(&self->super); - log_source_set_options(&self->super, options, 0, SCS_INTERNAL, owner->super.id, NULL, FALSE); + log_source_set_options(&self->super.super, options, 0, SCS_INTERNAL, owner->super.super.id, NULL, FALSE); afinter_source_init_watches(self); self->super.super.init = afinter_source_init; self->super.super.deinit = afinter_source_deinit; @@ -328,7 +328,7 @@ afinter_sd_init(LogPipe *s) return FALSE; } - log_source_options_init(&self->source_options, cfg, self->super.group); + log_source_options_init(&self->source_options, cfg, self->super.super.group); self->source = afinter_source_new(self, &self->source_options); log_pipe_append(&self->source->super, s); log_pipe_init(&self->source->super, cfg); @@ -370,9 +370,9 @@ afinter_sd_new(void) AFInterSourceDriver *self = g_new0(AFInterSourceDriver, 1); log_src_driver_init_instance((LogSrcDriver *)&self->super); - self->super.super.init = afinter_sd_init; - self->super.super.deinit = afinter_sd_deinit; - self->super.super.free_fn = afinter_sd_free; + self->super.super.super.init = afinter_sd_init; + self->super.super.super.deinit = afinter_sd_deinit; + self->super.super.super.free_fn = afinter_sd_free; log_source_options_defaults(&self->source_options); return (LogDriver *)&self->super.super; } diff --git a/lib/afinter.h b/lib/afinter.h index 108475f..1cd9d78 100644 --- a/lib/afinter.h +++ b/lib/afinter.h @@ -33,7 +33,7 @@ */ typedef struct _AFInterSourceDriver { - LogDriver super; + LogSrcDriver super; LogSource *source; LogSourceOptions source_options; } AFInterSourceDriver; -- 1.7.9.1

2 1

ts_format(iso) bug or misunderstanding?
by Chris Hiestand 01 May '12

01 May '12

So I'm using iso timezone format for my syslog clients: options { … ts_format(iso); … }; I expected this to use the iso format for all syslog entries. However, I've found that it only uses iso format for local syslog entries. syslog entries sent to the syslog-ng collector are in old rfc3164 format. > chiestand@host:~$ logger -t my-test asdlfkjasdfasdf which sends (3 local entries, 1 remote entry): > sudo strace -ff -p 13658 > ... > write(10, "2012-04-05T18:14:37-07:00 host "..., 64) = 64 > write(11, "2012-04-05T18:14:37-07:00 host "..., 64) = 64 > write(9, "<13>Apr 5 18:25:09 host my-test"..., 58) = 58 > write(14, "2012-04-05T18:14:37-07:00 host "..., 64) = 64 or viewing with tcpdump: > chiestand@host:/var/log$ sudo tcpdump -A host syslog.server.salk.edu > … > .va<13>Apr 5 18:20:31 host my-test: asdlfkjasdfasdf I would expect the iso format to be sent to the syslog-ng collector as well. Is this a bug or expected behavior? I checked bugzilla and debian bug tracker and didn't see anything. I'm running syslog-ng v3.1 on Debian squeeze. Thanks, Chris

3 5

syslog-ng 3.4.0alpha2 has been released
by devel＠balabit.hu 01 May '12

01 May '12

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.4.0alpha2 SUMMARY : new stable release DATE : May 1, 2012 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.4.0alpha2) has been released. For latest fixes in the 3.4.x feature branch you are recommended to upgrade to this version. CHANGES: 3.4.0alpha2 Tue, 01 May 2012 08:59:21 +0200 Features: ========= * Added support for globbing when including files. Now you can use UNIX shell style wildcards in the names of include files and syslog-ng will properly source all matching files. * Allow huge messages. Earlier log messages were limited to 256k in size, with each name-value pair limited at 64k. These limitations were lifted, syslog-ng now uses a 32 bit integer field to hold the size information for both the whole data structure and individual fields. An artificial limit of 256MB is still applied though, but it is trivial to change that in the future, and until then this should suffice. Other changes: ============== * Flow-control logic was changed, instead of enabling flow-control for all branches of a log statement when a single flags(flow-control) was present, only enable it downwards. * Changed json-parser() to encode JSON array members as ${array[0]}, ${array[1]} and so on, instead of using ".index" suffixes. * Fixed Makefile.am files to avoid attempting to compile mongodb and smtp destination drivers when they are disabled. * Fixed --with-json configure option parsing and added --disable-json for consistency with other optional features. * Removed the "dummy" module, which only served as a placeholder when the original "pluginization" happened in 3.2. Bugfixes: ========= * Fixed a segmentation fault, which happened when source or destination objects were defined without any actual drivers. * Fixed a possible crash in config file parsing that happens when parsing a unix-dgram() source driver and file permissions (owner, group, perm) specified. * Fixed a possible crash (double-free) in the value-pairs command line parser used in $(format-json), in case the command line options have syntax errors. * Handle and print template compilation errors properly during configuration parsing. * Fixed some memory leaks in the configuration compilation code introduced in 3.4. Bugfixes backported from 3.3 (matching 3.3.5): ============================================== * Fixed a significant memory leak that occurred when the receiver side of a syslog connection is slower than syslog-ng. The message which we got EAGAIN for got leaked. * Fixed another memory leak that caused the complete queue for socket based destination drivers to be leaked at reload time. * Fixed a memory leak related to handling TLS enabled connections. As connections come and go the TLS context wasn't freed. * Fixed a small memory leak that occurred when worker threads exit (and later start again). * Fixed a deadlock in the SQL destination that occurred when a COMMIT TRANSACTION command failed. * Fixed an abort() which occurred when the retry timer of a suspended destination fired. * Fixed a framing error in RFC5424 destinations. * MongoDB & SQL: fixed a race condition that caused mongodb/sql destinations to hang. * Fixed handling the ENOBUFS error in udp() destinations on FreeBSD and related/derived platforms. Previously ENOBUFS caused syslog-ng to suspend the destination for time_reopen() (which defaults to 60 seconds), but that's excessive. Reading the literature suggests that there are three ways to handle an ENOBUFS: wait for some undetermined amount of time until the interface buffers are presumably emptied, or let the packet be dropped or go back to the poll loop and hope that by the next time the event is dispatched ENOBUFS will be gone. Right now, syslog-ng will drop the affected packet, as the other two could cause other tasks to starve (wait some time) OR syslog-ng to use 100% CPU. Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Peter Czanik (BalaBit) Martin Grauel (BalaBit) Patrick Hemmer Hendrik Völker (Verizon) Gergely Nagy (BalaBit) Evan Rempel (University of Victoria) Balint Kovacs (BalaBit) Heiko Gerstung Andreas Piesk Matthias Runge (Fedora) DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

syslog-ng 3.3.5 has been released
by devel＠balabit.hu 01 May '12

01 May '12

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.3.5 SUMMARY : new stable release DATE : May 1, 2012 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.3.5) has been released. For latest fixes in the 3.3.x feature branch you are recommended to upgrade to this version. CHANGES: 3.3.5 Sat, 07 Apr 2012 10:35:25 +0200 Bugfixes: ========= * Fixed a significant memory leak that occurred when the receiver side of a syslog connection is slower than syslog-ng. The message which we got EAGAIN for got leaked. * Fixed another memory leak that caused the complete queue for socket based destination drivers to be leaked at reload time. * Fixed a memory leak related to handling TLS enabled connections. As connections come and go the TLS context wasn't freed. * Fixed a small memory leak that occurred when worker threads exit (and later start again). * Fixed a deadlock in the SQL destination that occurred when a COMMIT TRANSACTION command failed. * Fixed an abort() which occurred when the retry timer of a suspended destination fired. * Fixed a framing error in RFC5424 destinations. * MongoDB & SQL: fixed a race condition that caused mongodb/sql destinations to hang. * Fixed handling the ENOBUFS error in udp() destinations on FreeBSD and related/derived platforms. Previously ENOBUFS caused syslog-ng to suspend the destination for time_reopen() (which defaults to 60 seconds), but that's excessive. Reading the literature suggests that there are three ways to handle an ENOBUFS: wait for some undetermined amount of time until the interface buffers are presumably emptied, or let the packet be dropped or go back to the poll loop and hope that by the next time the event is dispatched ENOBUFS will be gone. Right now, syslog-ng will drop the affected packet, as the other two could cause other tasks to starve (wait some time) OR syslog-ng to use 100% CPU. Other changes: ============== * The XML sources for manual pages are now included in the source tarball, along with a set of already built man pages to avoid having to install XML related toolings to compile syslog-ng. The path to the stylesheets and the path to xsltproc is hard-coded in the Makefiles right now (to the paths used by Ubuntu 11.10) In case you want to change and rebuild manpages these need to be adjusted. Patches to autodetect the proper paths on any system would be welcome. * Rewrite expressions report an error during the configuration parsing phase instead of silently (ok with a warning :) falling back to changing the MESSAGE macro. Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Balazs Scheidler (BalaBit) Eric Lindvall (SevenScale) Evan Rempel (University of Victoria) Fried Zoltan (BalaBit) Gergely Nagy (BalaBit) Jakub Jankowski (SuperHost.pl) Juhasz Viktor (BalaBit) Marvin Nipper (Stream Global Services) Patrick Hemmer Robert Fekete (BalaBit) Traiano Welcome whille Yuri Arabadji (Fused) DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

Spoofed source bug introduced in 3.3.5
by Marvin Nipper 30 Apr '12

30 Apr '12

Hi. All of this pertains to a Solaris 10 (x86) environment.... I tried, at the end of the week, to jump up to the new 3.3.5 release, mostly to get up to a version that already included all of the memory leak fixes that I had manually deployed to 3.3.4. It all compiled fine, and seemed to work fine.... Until this weekend. When my log rotation script ran this weekend, all hell broke loose. The specific issue is that the instant that syslog-ng 3.3.5 receives the HUP (from my rotation script), all of the packets that are being forwarded (via UDP, with source spoofing) instantly start being forwarded with null (0.0.0.0) destination addresses. (And, FYI, I can easily recreate, and see this behavior.) Unfortunately, instead of _only_ resulting in the destination server not receiving these packets, the effect is actually far worse, because a null destination address is (at least on Solaris) interpreted as an old form of a broadcast packet, and all of these packets actually create an exponential feedback loop. I.E. they are seen by this same syslog-ng server's UDP listener, as valid inbound packets, which (of course), then are forwarded, outbound (by syslog-ng, as part of the spoofed forwarding), which then (of course) causes them to be seen (yet again) by syslog-ng as inbound packets. And "very quickly" the whole thing piles up on itself, causing the server to pretty much collapse under the weight of an ever increasing volume of "perceived" inbound UDP syslog packets. So... I'm hoping that someone will say "ah ha", and have some idea about what code might have changed between 3.3.4 and 3.3.5, that might have (accidentally) resulted in this behavior. Certainly, I've dropped back (for now) to 3.3.4 (as it does not have the problem), but if there is some sort of debugging that you need me to do (i.e. "if it's not obvious what coding change may have introduced this bug"), I'm more than willing to deploy the 3.3.5 code in a test environment, and do whatever testing that you might need. Again, it's easily reproducible with a simple HUP. Sorry to bring you a new/different issue. Any and all input/help would be appreciated. As always, THANKS for your help!! Marvin Nipper This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

2 2