May 2011 - syslog-ng - lists.balabit.hu

PCRE subst example from documentation produces error
by Daniel Maher 02 May '11

02 May '11

Hello, While following the example given here : http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.2-gui… I copied and pasted the string, then attempted to restart the daemon; unfortunately, this was the result : --- Error parsing rewrite expression, syntax error, unexpected LL_IDENTIFIER, expecting ')' in /etc/syslog-ng/syslog-ng.conf at line 354, column 27: {subst("a*", "?", field("message") type("pcre") flags("utf8" "global")); }; ^^^^^ syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng Cannot load /etc/syslog-ng/syslog-ng.conf --- Have I managed to paste incorrectly, or is the documentation incorrect, or is it something else entirely ? # syslog -V syslog-ng 3.2.2 Installer-Version: 3.2.2 Revision: Compile-Date: Apr 20 2011 16:19:28 Enable-Threads: off Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: off Enable-SSL: on Enable-SQL: off Enable-Linux-Caps: off Enable-Pcre: on Enable-Pacct: off Thank you. -- Daniel Maher « can't talk, too busy calculating computrons. »

2 1

[PATCH 0/3]: value-pairs() & related patches
by Gergely Nagy 01 May '11

01 May '11

Hi! For a multitude of reasons, ranging from the feature freeze looming over my head to hope that discrete patches are easier to review and merge than my awfully chaotic git tree, I'll be posting a couple of patches in a short while, as soon as I figure out what magic arguments I need to give to git send-email to make it send the batch the way I imagined. value-pairs() ============= The first one will be the value-pairs() implementation (also available on my integration/value-pairs branch), which is the base of the two others that will follow. While the value-pairs() implementation could still use some work, especially in the performance department, I believe it's good enough for initial use, and the benefits of having it (so that the mongodb driver can make use of it) far outweight the downsides. The implementation itself is reasonably light, it hardly touches anything outside it's own two files: the grammar and the makefile only, to hook itself in. If not used by a driver, then it does not change functionality at all. Therefore, I believe this could go in before the feature freeze. afmongodb value-pairs() port ============================ The next patch will be the port of the already existing mongodb destination driver to use value-pairs(). Also available on my integration/afmongodb-vp branch. This means that the driver will finally be able to do what it's good at: store arbitrary information in the database: whether that information comes via SDATA or patterndb, it doesn't matter, it will be stored in a structured manner. In my opinion, this is the major selling point of the mongodb destination driver at the moment (I have some other grand plans with it in the future, but that's more invasive, less important, and can easily wait 'till 3.4 ;). Without value-pairs(), afmongodb is pretty much useless. tfson() ======= For extra fun & profit, Balint Kovacs' format_json has been ported to use value-pairs(), and this will be the third patch in the series, and is also available on my integration/tfjson branch. This one's the least important, though it's dead simple, therefore I'd be very happy if it made past the feature freeze aswell. -- |8]

3 15

syslog-ng 3.2.3 has been released
by devel＠balabit.hu 01 May '11

01 May '11

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.2.3 SUMMARY : new stable release DATE : May 1, 2011 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.2.3) has been released. For latest fixes in the 3.2.x feature branch you are recommended to upgrade to this version. CHANGES: 3.2.3 Sun, 01 May 2011 19:05:32 +0200 Highlights: This is a maintenance release for the 3.2 branch, which contains several important functionality fixes in the db-parser() correllation code, an important security fix for FreeBSD & HP-UX (CVE-2011-0343) and build fixes for cygwin and mixed mode linking. Security fixes: * Fixed a possible security issue on Debian/kFreeBSD and on platforms where mode_t is an unsigned 16 bit value (FreeBSD, HP-UX). On these platforms syslog-ng may be using 0xFFFF as the permission bits. (CVE-2011-0343) Bugfixes: * Fixes an y2k38 problem that causes syslog-ng to use 100% CPU time in case mark messages are enabled and the UNIX timestamps overflows a signed 32 bit counter, which happens in 19th, January 2038 * Fixed file() destination to work on device nodes (e.g. files in /dev). Without this change, syslog-ng started using 100% CPU time if given devices as destinations that couldn't always consume data. * The code to restore the last file position for source files will not accept file-position past the file size, and will restart the file from the beginning instead. * Don't attempt to remember the current file position for source files that are read with follow-freq(0), e.g. /dev/klog and /proc/kmsg. These are special files which do not have the notion of file position, so no need to remember them. Regular files should always be read with follow-freq() set to nonzero, which is the default. * Fixed linking unit tests and other tools in mixed linking mode. * Fixed compilation on cygwin, especially lot of efforts went into linker compatibility. * Fixed building on platforms where PCRE is not in the standard include path. * Accept catch-all flag on log statements as well as catchall, as this was incorrectly documented in the past. db-parser() & pdbtool bugfixes: * Fixed @XX style message reference parsing used in correllation rules. * Fixed a segfault in the $(grep) template function when processing the parameters failed. * Fixed segfault in "pdbtool match --debug-pattern" in case the pattern doesn't match. * Fixed "pdbtool test" as previously all patterndb.xml files were reported to be invalid, even valid ones. * Fixed correllation timer related issue that caused some timers not to expire in rare cases. db-parser() & pdbtool changes: * Added support for enclosing template function arguments in parenthesis, in which case the quotes within the parentheses are not removed. For example: $(grep ('$FACILITY' == 'syslog')) This makes writing $(grep) and $(if) arguments much easier. * dbparser() the @NUMBER@ and @FLOAT@ parsers are able to parse negative numbers. * Added debug messages to dbparser() correllation so that it becomes easier to diagnose db-parser() problems. * Added -d (for --debug) and -v (for --verbose) options to pdbtool in order to make patterndb debugging easier. * Added --no-parse option to "pdbtool patternize" in order to read files without syslog-style parsing. Other changes: * Added error messages on DBI initialization failures. * Added systemd socket activation support. Build related changes: * Support for old (e.g. pre 7.1 commonly found in RHEL5) PCRE versions at the cost of an inoperating "newline" regexp flag. * configure now validates flex/bison versions better, as the requirements are more strict starting with syslog-ng 3.2 * Drop the creation of libsyslog-ng-patterndb.so. * "make clean" will properly remove libafsocket.so symlink. Credits: syslog-ng is developed as a community project. All changes and improvements requires effort, and this effort is really appreciated. Writing code, testing changes or simply providing use-cases and information on one's setup will make syslog-ng better. Here are the people, listed in no specific order who made this release possible. Steven Chamberlain Roger Paul Krizak (AMD) Attila Szalay (BalaBit) Matthew Hall Marius Tomaschewski (SUSE) Gergely Nagy (BalaBit) Tom Gundersen Dalibor Toman (Fortech.cz) Corinna Vinschen (RedHat) Balazs Scheidler (BalaBit) Laszlo Boszormenyi (LSC.hu) Arkadiusz Miśkiewicz (PLD Linux) Valentijn Sessink Hendrik Visage Peter Gyongyosi (BalaBit) Zoltan Pallagi (BalaBit) Mishou Michael (US IRS) Thanks for their efforts, it is appreciated. DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

[PATCH] basicfuncs: Implement a $(substr START LEN STR) template function.
by Gergely Nagy 01 May '11

01 May '11

This implements the $(substr) template function - a fairly primitive one: it does error checking, but there's no error reporting present, therefore invalid function calls will just get ignored. Usage should be straightforward. Reported-By: Jakub Jankowski <shasta(a)toxcorp.com> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/basicfuncs/basic-funcs.c | 46 ++++++++++++++++++++++++++++++++++++++ 1 files changed, 46 insertions(+), 0 deletions(-) diff --git a/modules/basicfuncs/basic-funcs.c b/modules/basicfuncs/basic-funcs.c index 44b8af3..542c2ec 100644 --- a/modules/basicfuncs/basic-funcs.c +++ b/modules/basicfuncs/basic-funcs.c @@ -4,6 +4,9 @@ #include "filter-expr-parser.h" #include "cfg.h" +#include <stdlib.h> +#include <errno.h> + static void tf_echo(LogMessage *msg, gint argc, GString *argv[], GString *result) { @@ -19,6 +22,48 @@ tf_echo(LogMessage *msg, gint argc, GString *argv[], GString *result) TEMPLATE_FUNCTION_SIMPLE(tf_echo); +static gboolean +tf_substr_parse_int(const gchar *s, long *d) +{ + gchar *endptr; + glong val; + + errno = 0; + val = strtoll(s, &endptr, 10); + + if ((errno == ERANGE && (val == LONG_MAX || val == LONG_MIN)) + || (errno != 0 && val == 0)) + return FALSE; + + if (endptr == s || *endptr != '\0') + return FALSE; + + *d = val; + return TRUE; +} + +static void +tf_substr(LogMessage *msg, gint argc, GString *argv[], GString *result) +{ + glong start, len; + + if (argc != 3) + return; + + if (!tf_substr_parse_int (argv[0]->str, &start)) + return; + if (!tf_substr_parse_int (argv[1]->str, &len)) + return; + + if (start > argv[2]->len || + start + len >= argv[2]->len) + return; + + g_string_append_len (result, argv[2]->str + start - 1, len); +} + +TEMPLATE_FUNCTION_SIMPLE(tf_substr); + typedef struct _TFCondState { FilterExprNode *filter; @@ -158,6 +203,7 @@ static Plugin basicfuncs_plugins[] = TEMPLATE_FUNCTION_PLUGIN(tf_echo, "echo"), TEMPLATE_FUNCTION_PLUGIN(tf_grep, "grep"), TEMPLATE_FUNCTION_PLUGIN(tf_if, "if"), + TEMPLATE_FUNCTION_PLUGIN(tf_substr, "substr"), }; gboolean -- 1.7.2.5

3 8