- syslog-ng - lists.balabit.hu

[PATCH] afinter: Make the parent class LogSrcDriver, instead of LogDriver.
by Gergely Nagy 01 May '12

01 May '12

The parent of AFInterSourceDriver should be LogSrcDriver, because that's the struct we pass down to it anyway, and other parts of the code will cast it back to LogSrcDriver, too. But if its parent is LogDriver, as it was, we were corrupting the later parts of the original LogSrcDriver, with values from AFInterSourceDriver. Many thanks to Peter Eisenlohr <peter.eisenlohr(a)inform-software.com> for initial debugging, and for pinpointing mark messages as the triggers. Reported-by: Peter Czanik <czanik(a)balabit.hu> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- lib/afinter.c | 10 +++++----- lib/afinter.h | 2 +- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/lib/afinter.c b/lib/afinter.c index 23b1c82..b9d0ddf 100644 --- a/lib/afinter.c +++ b/lib/afinter.c @@ -304,7 +304,7 @@ afinter_source_new(AFInterSourceDriver *owner, LogSourceOptions *options) AFInterSource *self = g_new0(AFInterSource, 1); log_source_init_instance(&self->super); - log_source_set_options(&self->super, options, 0, SCS_INTERNAL, owner->super.id, NULL, FALSE); + log_source_set_options(&self->super.super, options, 0, SCS_INTERNAL, owner->super.super.id, NULL, FALSE); afinter_source_init_watches(self); self->super.super.init = afinter_source_init; self->super.super.deinit = afinter_source_deinit; @@ -328,7 +328,7 @@ afinter_sd_init(LogPipe *s) return FALSE; } - log_source_options_init(&self->source_options, cfg, self->super.group); + log_source_options_init(&self->source_options, cfg, self->super.super.group); self->source = afinter_source_new(self, &self->source_options); log_pipe_append(&self->source->super, s); log_pipe_init(&self->source->super, cfg); @@ -370,9 +370,9 @@ afinter_sd_new(void) AFInterSourceDriver *self = g_new0(AFInterSourceDriver, 1); log_src_driver_init_instance((LogSrcDriver *)&self->super); - self->super.super.init = afinter_sd_init; - self->super.super.deinit = afinter_sd_deinit; - self->super.super.free_fn = afinter_sd_free; + self->super.super.super.init = afinter_sd_init; + self->super.super.super.deinit = afinter_sd_deinit; + self->super.super.super.free_fn = afinter_sd_free; log_source_options_defaults(&self->source_options); return (LogDriver *)&self->super.super; } diff --git a/lib/afinter.h b/lib/afinter.h index 108475f..1cd9d78 100644 --- a/lib/afinter.h +++ b/lib/afinter.h @@ -33,7 +33,7 @@ */ typedef struct _AFInterSourceDriver { - LogDriver super; + LogSrcDriver super; LogSource *source; LogSourceOptions source_options; } AFInterSourceDriver; -- 1.7.9.1

2 1

ts_format(iso) bug or misunderstanding?
by Chris Hiestand 01 May '12

01 May '12

So I'm using iso timezone format for my syslog clients: options { … ts_format(iso); … }; I expected this to use the iso format for all syslog entries. However, I've found that it only uses iso format for local syslog entries. syslog entries sent to the syslog-ng collector are in old rfc3164 format. > chiestand@host:~$ logger -t my-test asdlfkjasdfasdf which sends (3 local entries, 1 remote entry): > sudo strace -ff -p 13658 > ... > write(10, "2012-04-05T18:14:37-07:00 host "..., 64) = 64 > write(11, "2012-04-05T18:14:37-07:00 host "..., 64) = 64 > write(9, "<13>Apr 5 18:25:09 host my-test"..., 58) = 58 > write(14, "2012-04-05T18:14:37-07:00 host "..., 64) = 64 or viewing with tcpdump: > chiestand@host:/var/log$ sudo tcpdump -A host syslog.server.salk.edu > … > .va<13>Apr 5 18:20:31 host my-test: asdlfkjasdfasdf I would expect the iso format to be sent to the syslog-ng collector as well. Is this a bug or expected behavior? I checked bugzilla and debian bug tracker and didn't see anything. I'm running syslog-ng v3.1 on Debian squeeze. Thanks, Chris

3 5

syslog-ng 3.4.0alpha2 has been released
by devel＠balabit.hu 01 May '12

01 May '12

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.4.0alpha2 SUMMARY : new stable release DATE : May 1, 2012 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.4.0alpha2) has been released. For latest fixes in the 3.4.x feature branch you are recommended to upgrade to this version. CHANGES: 3.4.0alpha2 Tue, 01 May 2012 08:59:21 +0200 Features: ========= * Added support for globbing when including files. Now you can use UNIX shell style wildcards in the names of include files and syslog-ng will properly source all matching files. * Allow huge messages. Earlier log messages were limited to 256k in size, with each name-value pair limited at 64k. These limitations were lifted, syslog-ng now uses a 32 bit integer field to hold the size information for both the whole data structure and individual fields. An artificial limit of 256MB is still applied though, but it is trivial to change that in the future, and until then this should suffice. Other changes: ============== * Flow-control logic was changed, instead of enabling flow-control for all branches of a log statement when a single flags(flow-control) was present, only enable it downwards. * Changed json-parser() to encode JSON array members as ${array[0]}, ${array[1]} and so on, instead of using ".index" suffixes. * Fixed Makefile.am files to avoid attempting to compile mongodb and smtp destination drivers when they are disabled. * Fixed --with-json configure option parsing and added --disable-json for consistency with other optional features. * Removed the "dummy" module, which only served as a placeholder when the original "pluginization" happened in 3.2. Bugfixes: ========= * Fixed a segmentation fault, which happened when source or destination objects were defined without any actual drivers. * Fixed a possible crash in config file parsing that happens when parsing a unix-dgram() source driver and file permissions (owner, group, perm) specified. * Fixed a possible crash (double-free) in the value-pairs command line parser used in $(format-json), in case the command line options have syntax errors. * Handle and print template compilation errors properly during configuration parsing. * Fixed some memory leaks in the configuration compilation code introduced in 3.4. Bugfixes backported from 3.3 (matching 3.3.5): ============================================== * Fixed a significant memory leak that occurred when the receiver side of a syslog connection is slower than syslog-ng. The message which we got EAGAIN for got leaked. * Fixed another memory leak that caused the complete queue for socket based destination drivers to be leaked at reload time. * Fixed a memory leak related to handling TLS enabled connections. As connections come and go the TLS context wasn't freed. * Fixed a small memory leak that occurred when worker threads exit (and later start again). * Fixed a deadlock in the SQL destination that occurred when a COMMIT TRANSACTION command failed. * Fixed an abort() which occurred when the retry timer of a suspended destination fired. * Fixed a framing error in RFC5424 destinations. * MongoDB & SQL: fixed a race condition that caused mongodb/sql destinations to hang. * Fixed handling the ENOBUFS error in udp() destinations on FreeBSD and related/derived platforms. Previously ENOBUFS caused syslog-ng to suspend the destination for time_reopen() (which defaults to 60 seconds), but that's excessive. Reading the literature suggests that there are three ways to handle an ENOBUFS: wait for some undetermined amount of time until the interface buffers are presumably emptied, or let the packet be dropped or go back to the poll loop and hope that by the next time the event is dispatched ENOBUFS will be gone. Right now, syslog-ng will drop the affected packet, as the other two could cause other tasks to starve (wait some time) OR syslog-ng to use 100% CPU. Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Peter Czanik (BalaBit) Martin Grauel (BalaBit) Patrick Hemmer Hendrik Völker (Verizon) Gergely Nagy (BalaBit) Evan Rempel (University of Victoria) Balint Kovacs (BalaBit) Heiko Gerstung Andreas Piesk Matthias Runge (Fedora) DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

syslog-ng 3.3.5 has been released
by devel＠balabit.hu 01 May '12

01 May '12

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.3.5 SUMMARY : new stable release DATE : May 1, 2012 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.3.5) has been released. For latest fixes in the 3.3.x feature branch you are recommended to upgrade to this version. CHANGES: 3.3.5 Sat, 07 Apr 2012 10:35:25 +0200 Bugfixes: ========= * Fixed a significant memory leak that occurred when the receiver side of a syslog connection is slower than syslog-ng. The message which we got EAGAIN for got leaked. * Fixed another memory leak that caused the complete queue for socket based destination drivers to be leaked at reload time. * Fixed a memory leak related to handling TLS enabled connections. As connections come and go the TLS context wasn't freed. * Fixed a small memory leak that occurred when worker threads exit (and later start again). * Fixed a deadlock in the SQL destination that occurred when a COMMIT TRANSACTION command failed. * Fixed an abort() which occurred when the retry timer of a suspended destination fired. * Fixed a framing error in RFC5424 destinations. * MongoDB & SQL: fixed a race condition that caused mongodb/sql destinations to hang. * Fixed handling the ENOBUFS error in udp() destinations on FreeBSD and related/derived platforms. Previously ENOBUFS caused syslog-ng to suspend the destination for time_reopen() (which defaults to 60 seconds), but that's excessive. Reading the literature suggests that there are three ways to handle an ENOBUFS: wait for some undetermined amount of time until the interface buffers are presumably emptied, or let the packet be dropped or go back to the poll loop and hope that by the next time the event is dispatched ENOBUFS will be gone. Right now, syslog-ng will drop the affected packet, as the other two could cause other tasks to starve (wait some time) OR syslog-ng to use 100% CPU. Other changes: ============== * The XML sources for manual pages are now included in the source tarball, along with a set of already built man pages to avoid having to install XML related toolings to compile syslog-ng. The path to the stylesheets and the path to xsltproc is hard-coded in the Makefiles right now (to the paths used by Ubuntu 11.10) In case you want to change and rebuild manpages these need to be adjusted. Patches to autodetect the proper paths on any system would be welcome. * Rewrite expressions report an error during the configuration parsing phase instead of silently (ok with a warning :) falling back to changing the MESSAGE macro. Credits: ======== syslog-ng is developed as a community project, and as such it relies on volunteers to do the work necessarily to produce syslog-ng. Reporting bugs, testing changes, writing code or simply providing feedback are all important contributions, so please if you are a user of syslog-ng, contribute. These people have helped in this release: Balazs Scheidler (BalaBit) Eric Lindvall (SevenScale) Evan Rempel (University of Victoria) Fried Zoltan (BalaBit) Gergely Nagy (BalaBit) Jakub Jankowski (SuperHost.pl) Juhasz Viktor (BalaBit) Marvin Nipper (Stream Global Services) Patrick Hemmer Robert Fekete (BalaBit) Traiano Welcome whille Yuri Arabadji (Fused) DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

Spoofed source bug introduced in 3.3.5
by Marvin Nipper 30 Apr '12

30 Apr '12

Hi. All of this pertains to a Solaris 10 (x86) environment.... I tried, at the end of the week, to jump up to the new 3.3.5 release, mostly to get up to a version that already included all of the memory leak fixes that I had manually deployed to 3.3.4. It all compiled fine, and seemed to work fine.... Until this weekend. When my log rotation script ran this weekend, all hell broke loose. The specific issue is that the instant that syslog-ng 3.3.5 receives the HUP (from my rotation script), all of the packets that are being forwarded (via UDP, with source spoofing) instantly start being forwarded with null (0.0.0.0) destination addresses. (And, FYI, I can easily recreate, and see this behavior.) Unfortunately, instead of _only_ resulting in the destination server not receiving these packets, the effect is actually far worse, because a null destination address is (at least on Solaris) interpreted as an old form of a broadcast packet, and all of these packets actually create an exponential feedback loop. I.E. they are seen by this same syslog-ng server's UDP listener, as valid inbound packets, which (of course), then are forwarded, outbound (by syslog-ng, as part of the spoofed forwarding), which then (of course) causes them to be seen (yet again) by syslog-ng as inbound packets. And "very quickly" the whole thing piles up on itself, causing the server to pretty much collapse under the weight of an ever increasing volume of "perceived" inbound UDP syslog packets. So... I'm hoping that someone will say "ah ha", and have some idea about what code might have changed between 3.3.4 and 3.3.5, that might have (accidentally) resulted in this behavior. Certainly, I've dropped back (for now) to 3.3.4 (as it does not have the problem), but if there is some sort of debugging that you need me to do (i.e. "if it's not obvious what coding change may have introduced this bug"), I'm more than willing to deploy the 3.3.5 code in a test environment, and do whatever testing that you might need. Again, it's easily reproducible with a simple HUP. Sorry to bring you a new/different issue. Any and all input/help would be appreciated. As always, THANKS for your help!! Marvin Nipper This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

2 2

Pattern Matching Help
by Rob Cameron 27 Apr '12

27 Apr '12

Team syslog-ng, I am attempting to write a parser to break apart a structured syslog message and break it into values that I will ultimately forward to MongoDB. I am unable to get the parsing to work. Below is a link to my parser database file and I have built in a simple test case. If someone could assist me by matching at least one field of the test log found in my example PDB I would be most appreciative for the help. I will also contribute my PDB back to the community. https://github.com/RobWC/syslog-ng-SRX/blob/master/junos-sme-12.1.pdb Thanks again for your help. -- Rob

2 1

Glib libraries issue syslog_ng 3.3.4
by Rakesh Rajasekharan 27 Apr '12

27 Apr '12

Hello Guys, I am trying to compile syslog_ng on solaris 10 opteron with mixed linking option. I was able to successfully compile glib-2.16.5 statically. in my configure stage of syslog_ng 3.3.4 , I get he below error. configure: error: static GLib libraries not found (a file named libglib-2.0.a), either link GLib dynamically using the --enable-dynamic-linking or install a static GLib libglib-2.0.a is present under /usr/local/lib . When I checked the config.log, I have the following errors. configure:5326: checking lex library configure:5340: gcc -std=gnu99 -o conftest -g -O2 conftest.c >&5 Undefined first referenced symbol in file yywrap /var/tmp//cc2KbW8B.o ld: fatal: Symbol referencing errors. No output written to conftest collect2: ld returned 1 exit status configure:5340: $? = 1 configure: failed program was: | /* confdefs.h */ | #define PACKAGE_NAME "" | #define PACKAGE_TARNAME "" | #define PACKAGE_VERSION "" | #define PACKAGE_STRING "" | #define PACKAGE_BUGREPORT "" | #define PACKAGE_URL "" | /* end confdefs.h. */ | | #line 3 "lex.yy.c" | | #define YY_INT_ALIGNED short int | | /* A lexical scanner generat Any inputs on how can i fix this please? Best Regards, Rakesh

2 4

(no subject)
by pedro castro 26 Apr '12

26 Apr '12

Single m0m makes real m0ney w0rking fr0m h0me http://cgm-web.com/pertmoun.php?bunpagge=38 Thu, 26 Apr 2012 20:44:00 ______________ " We shan't see a bit of it, worse luck! Nota sniff we shan't get!2The naval situation in the North Atlantic at that time was apeculiar one" (c) adalberto yule

1 0

afsql error
by Jason Kojro-Badziak 23 Apr '12

23 Apr '12

Hello all: I am running syslog-ng on a Centos 6 server. I installed syslog-ng via yum, and while it is running successfully, I am getting this error every time I start or restart syslog-ng. # /etc/init.d/syslog-ng start Plugin module not found in 'module-path'; module-path='/lib64/syslog-ng', module='afsql' Starting syslog-ng: Plugin module not found in 'module-path'; module-path='/lib64/syslog-ng', module='afsql' [ OK ] Is there something I can do to fix this so I stop getting the error? Thank you! Jason Kojro-Badziak Monolith Software Staff Engineer 311 North 2nd Street, Suite #302 St. Charles, IL 60174 Office: 312-957-6470 x3010 Email: jbadziak(a)monolith-software.com<mailto:jbadziak@monolith-software.com/jb.022411>

3 5

[Bug 156] New: rewrite rule not working on Linux
by bugzilla＠bugzilla.balabit.com 20 Apr '12

20 Apr '12

https://bugzilla.balabit.com/show_bug.cgi?id=156 Summary: rewrite rule not working on Linux Product: syslog-ng Version: 3.3.x Platform: Other OS/Version: Linux Status: NEW Severity: major Priority: unspecified Component: syslog-ng AssignedTo: bazsi(a)balabit.hu ReportedBy: ne24(a)georgetown.edu Type of the Report: bug Estimated Hours: 0.0 Hi, I have syslog-ng 3.3.3 running Red Hat Enterprise Linux Server release 6.2 (Santiago). I compiled with the following flags. ./configure --prefix=/usr/local --enable-spoof-source --enable-pcre --enable-ssl \ --enable-debug I have the following rewrite rule: rewrite n_dst_router { subst("^(%(?:ASA|PIX|FWSM)\-\d\-\d{6}):", "", value("MESSAGE"), type("pcre"), flags("global")); }; But it is not working. The line below minus the IP addresses shows that it is not replacing the %FWSM-6-305011 with a blank. Jan 7 15:59:06 HOSTNAME-REMOVED %FWSM-6-305011: Built dynamic tcp translation from GRT:IP-ADDRESS-REMOVED/53328 to BACKBONE-VRF:IP-ADDRESS-REMOVED/29361 I will also attach my whole syslog-ng.conf file so that you would see the whole setup. Here is the output of the debug output and as you can see, it looks like it is doing what it is supposed to, but it does not write it to the file with the change. Incoming log entry; line='<163>%FWSM-3-710003: udp access denied by ACL from IP-ADDRESS-REMOVED/35390 to BACKBONE-VRF:IP-ADDRESS-REMOVED/46897\x0a' Rewrite expression evaluation result; value='MESSAGE', new_value='udp access denied by ACL from IP-ADDRESS-REMOVED/35390 to BACKBONE-VRF:IP-ADDRESS-REMOVED/46897' I searched the bug reports but could not find anything related to this issue. I would appreciate it your help on this. Thank you Nadim El-Khoury -- Configure bugmail: https://bugzilla.balabit.com/userprefs.cgi?tab=email ------- You are receiving this mail because: ------- You are watching all bug changes.

1 2