- syslog-ng - lists.balabit.hu

Re: [syslog-ng] [PATCH] Use CAP_SYSLOG instead of CAP_SYS_ADMIN, if available.
by Gergely Nagy 02 May '11

02 May '11

Serge Hallyn <serge.hallyn(a)canonical.com> writes: > Quoting Balazs Scheidler (bazsi(a)balabit.hu) >> Hi, >> >> Thanks for the great stuff, it is really appreciated. >> >> I've some notes, please let me know if you or Serge would like to do >> that, or I should. > > Thanks, by all means please go ahead, your suggestions certainly sound > cleaner, and I'd probably end up producing something that ended up > requiring at least one more review cycle :) In that case, I'll have a round of fun with it (in a day or two, if all goes well). -- |8]

1 0

syslog-ng 3.2.3 build failure
by Sandor Geller 02 May '11

02 May '11

Hello, The below change breaks linking syslog-ng-ctl on OSes where the resolver library provides inet_aton so the -lresolv linker flag is needed. So far I ran into the issue only on Solaris 8 which isn't a modern OS I know... --- syslog-ng-3.2.2/syslog-ng/Makefile.in 2011-01-15 17:15:10.000000000 +0100 +++ syslog-ng-3.2.3/syslog-ng/Makefile.in 2011-05-01 21:47:51.000000000 +0200 @@ -60,7 +60,7 @@ syslog_ng_DEPENDENCIES = am_syslog_ng_ctl_OBJECTS = syslog-ng-ctl.$(OBJEXT) syslog_ng_ctl_OBJECTS = $(am_syslog_ng_ctl_OBJECTS) -syslog_ng_ctl_DEPENDENCIES = ../lib/libsyslog-ng.la +syslog_ng_ctl_DEPENDENCIES = ../lib/gsocket.lo ../lib/gsockaddr.lo am_syslog_ng_wrapper_OBJECTS = wrapper.$(OBJEXT) syslog_ng_wrapper_OBJECTS = $(am_syslog_ng_wrapper_OBJECTS) syslog_ng_wrapper_LDADD = $(LDADD) The fix seems to be trivial and hopefully doesn't have side-effects on any OS: --- syslog-ng-3.2.3/configure.in 2011-05-01 21:46:51.000000000 +0200 +++ syslog-ng-3.2.3-hacked/configure.in 2011-05-02 14:06:27.000000000 +0200 @@ -845,6 +845,7 @@ AC_SUBST(SOURCE_REVISION) AC_SUBST(PACKAGE_FILES_SUFFIX) AC_SUBST(PACKAGE_SUFFIX) +AC_SUBST(RESOLV_LIBS) AC_OUTPUT(dist.conf diff -urN syslog-ng-3.2.3/syslog-ng/Makefile.am syslog-ng-3.2.3-hacked/syslog-ng/Makefile.am --- syslog-ng-3.2.3/syslog-ng/Makefile.am 2011-02-23 11:15:06.000000000 +0100 +++ syslog-ng-3.2.3-hacked/syslog-ng/Makefile.am 2011-05-02 13:53:44.000000000 +0200 @@ -19,7 +19,7 @@ syslog_ng_wrapper_SOURCES = wrapper.c syslog_ng_ctl_SOURCES = syslog-ng-ctl.c -syslog_ng_ctl_LDADD = ../lib/gsocket.lo ../lib/gsockaddr.lo @BASE_LIBS@ @GLIB_LIBS@ +syslog_ng_ctl_LDADD = ../lib/gsocket.lo ../lib/gsockaddr.lo @BASE_LIBS@ @GLIB_LIBS@ @RESOLV_LIBS@ if ENABLE_ENV_WRAPPER Regards, Sandor

2 2

PCRE subst example from documentation produces error
by Daniel Maher 02 May '11

02 May '11

Hello, While following the example given here : http://www.balabit.com/sites/default/files/documents/syslog-ng-ose-v3.2-gui… I copied and pasted the string, then attempted to restart the daemon; unfortunately, this was the result : --- Error parsing rewrite expression, syntax error, unexpected LL_IDENTIFIER, expecting ')' in /etc/syslog-ng/syslog-ng.conf at line 354, column 27: {subst("a*", "?", field("message") type("pcre") flags("utf8" "global")); }; ^^^^^ syslog-ng documentation: http://www.balabit.com/support/documentation/?product=syslog-ng mailing list: https://lists.balabit.hu/mailman/listinfo/syslog-ng Cannot load /etc/syslog-ng/syslog-ng.conf --- Have I managed to paste incorrectly, or is the documentation incorrect, or is it something else entirely ? # syslog -V syslog-ng 3.2.2 Installer-Version: 3.2.2 Revision: Compile-Date: Apr 20 2011 16:19:28 Enable-Threads: off Enable-Debug: off Enable-GProf: off Enable-Memtrace: off Enable-Sun-STREAMS: off Enable-IPv6: on Enable-Spoof-Source: on Enable-TCP-Wrapper: off Enable-SSL: on Enable-SQL: off Enable-Linux-Caps: off Enable-Pcre: on Enable-Pacct: off Thank you. -- Daniel Maher « can't talk, too busy calculating computrons. »

2 1

[PATCH 0/3]: value-pairs() & related patches
by Gergely Nagy 01 May '11

01 May '11

Hi! For a multitude of reasons, ranging from the feature freeze looming over my head to hope that discrete patches are easier to review and merge than my awfully chaotic git tree, I'll be posting a couple of patches in a short while, as soon as I figure out what magic arguments I need to give to git send-email to make it send the batch the way I imagined. value-pairs() ============= The first one will be the value-pairs() implementation (also available on my integration/value-pairs branch), which is the base of the two others that will follow. While the value-pairs() implementation could still use some work, especially in the performance department, I believe it's good enough for initial use, and the benefits of having it (so that the mongodb driver can make use of it) far outweight the downsides. The implementation itself is reasonably light, it hardly touches anything outside it's own two files: the grammar and the makefile only, to hook itself in. If not used by a driver, then it does not change functionality at all. Therefore, I believe this could go in before the feature freeze. afmongodb value-pairs() port ============================ The next patch will be the port of the already existing mongodb destination driver to use value-pairs(). Also available on my integration/afmongodb-vp branch. This means that the driver will finally be able to do what it's good at: store arbitrary information in the database: whether that information comes via SDATA or patterndb, it doesn't matter, it will be stored in a structured manner. In my opinion, this is the major selling point of the mongodb destination driver at the moment (I have some other grand plans with it in the future, but that's more invasive, less important, and can easily wait 'till 3.4 ;). Without value-pairs(), afmongodb is pretty much useless. tfson() ======= For extra fun & profit, Balint Kovacs' format_json has been ported to use value-pairs(), and this will be the third patch in the series, and is also available on my integration/tfjson branch. This one's the least important, though it's dead simple, therefore I'd be very happy if it made past the feature freeze aswell. -- |8]

3 15

syslog-ng 3.2.3 has been released
by devel＠balabit.hu 01 May '11

01 May '11

------------------------------------------------------------------------------ PACKAGE : syslog-ng VERSION : 3.2.3 SUMMARY : new stable release DATE : May 1, 2011 ------------------------------------------------------------------------------ DESCRIPTION: A new stable version of syslog-ng Open Source Edition (3.2.3) has been released. For latest fixes in the 3.2.x feature branch you are recommended to upgrade to this version. CHANGES: 3.2.3 Sun, 01 May 2011 19:05:32 +0200 Highlights: This is a maintenance release for the 3.2 branch, which contains several important functionality fixes in the db-parser() correllation code, an important security fix for FreeBSD & HP-UX (CVE-2011-0343) and build fixes for cygwin and mixed mode linking. Security fixes: * Fixed a possible security issue on Debian/kFreeBSD and on platforms where mode_t is an unsigned 16 bit value (FreeBSD, HP-UX). On these platforms syslog-ng may be using 0xFFFF as the permission bits. (CVE-2011-0343) Bugfixes: * Fixes an y2k38 problem that causes syslog-ng to use 100% CPU time in case mark messages are enabled and the UNIX timestamps overflows a signed 32 bit counter, which happens in 19th, January 2038 * Fixed file() destination to work on device nodes (e.g. files in /dev). Without this change, syslog-ng started using 100% CPU time if given devices as destinations that couldn't always consume data. * The code to restore the last file position for source files will not accept file-position past the file size, and will restart the file from the beginning instead. * Don't attempt to remember the current file position for source files that are read with follow-freq(0), e.g. /dev/klog and /proc/kmsg. These are special files which do not have the notion of file position, so no need to remember them. Regular files should always be read with follow-freq() set to nonzero, which is the default. * Fixed linking unit tests and other tools in mixed linking mode. * Fixed compilation on cygwin, especially lot of efforts went into linker compatibility. * Fixed building on platforms where PCRE is not in the standard include path. * Accept catch-all flag on log statements as well as catchall, as this was incorrectly documented in the past. db-parser() & pdbtool bugfixes: * Fixed @XX style message reference parsing used in correllation rules. * Fixed a segfault in the $(grep) template function when processing the parameters failed. * Fixed segfault in "pdbtool match --debug-pattern" in case the pattern doesn't match. * Fixed "pdbtool test" as previously all patterndb.xml files were reported to be invalid, even valid ones. * Fixed correllation timer related issue that caused some timers not to expire in rare cases. db-parser() & pdbtool changes: * Added support for enclosing template function arguments in parenthesis, in which case the quotes within the parentheses are not removed. For example: $(grep ('$FACILITY' == 'syslog')) This makes writing $(grep) and $(if) arguments much easier. * dbparser() the @NUMBER@ and @FLOAT@ parsers are able to parse negative numbers. * Added debug messages to dbparser() correllation so that it becomes easier to diagnose db-parser() problems. * Added -d (for --debug) and -v (for --verbose) options to pdbtool in order to make patterndb debugging easier. * Added --no-parse option to "pdbtool patternize" in order to read files without syslog-style parsing. Other changes: * Added error messages on DBI initialization failures. * Added systemd socket activation support. Build related changes: * Support for old (e.g. pre 7.1 commonly found in RHEL5) PCRE versions at the cost of an inoperating "newline" regexp flag. * configure now validates flex/bison versions better, as the requirements are more strict starting with syslog-ng 3.2 * Drop the creation of libsyslog-ng-patterndb.so. * "make clean" will properly remove libafsocket.so symlink. Credits: syslog-ng is developed as a community project. All changes and improvements requires effort, and this effort is really appreciated. Writing code, testing changes or simply providing use-cases and information on one's setup will make syslog-ng better. Here are the people, listed in no specific order who made this release possible. Steven Chamberlain Roger Paul Krizak (AMD) Attila Szalay (BalaBit) Matthew Hall Marius Tomaschewski (SUSE) Gergely Nagy (BalaBit) Tom Gundersen Dalibor Toman (Fortech.cz) Corinna Vinschen (RedHat) Balazs Scheidler (BalaBit) Laszlo Boszormenyi (LSC.hu) Arkadiusz Miśkiewicz (PLD Linux) Valentijn Sessink Hendrik Visage Peter Gyongyosi (BalaBit) Zoltan Pallagi (BalaBit) Mishou Michael (US IRS) Thanks for their efforts, it is appreciated. DOWNLOAD: You can download the source or binary packages from: http://www.balabit.com/network-security/syslog-ng/opensource-logging-system… The documentation of the syslog-ng Open Source Edition is available in The syslog-ng Open Source Edition Administrator's Guide at http://www.balabit.com/support/documentation/

1 0

[PATCH] basicfuncs: Implement a $(substr START LEN STR) template function.
by Gergely Nagy 01 May '11

01 May '11

This implements the $(substr) template function - a fairly primitive one: it does error checking, but there's no error reporting present, therefore invalid function calls will just get ignored. Usage should be straightforward. Reported-By: Jakub Jankowski <shasta(a)toxcorp.com> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/basicfuncs/basic-funcs.c | 46 ++++++++++++++++++++++++++++++++++++++ 1 files changed, 46 insertions(+), 0 deletions(-) diff --git a/modules/basicfuncs/basic-funcs.c b/modules/basicfuncs/basic-funcs.c index 44b8af3..542c2ec 100644 --- a/modules/basicfuncs/basic-funcs.c +++ b/modules/basicfuncs/basic-funcs.c @@ -4,6 +4,9 @@ #include "filter-expr-parser.h" #include "cfg.h" +#include <stdlib.h> +#include <errno.h> + static void tf_echo(LogMessage *msg, gint argc, GString *argv[], GString *result) { @@ -19,6 +22,48 @@ tf_echo(LogMessage *msg, gint argc, GString *argv[], GString *result) TEMPLATE_FUNCTION_SIMPLE(tf_echo); +static gboolean +tf_substr_parse_int(const gchar *s, long *d) +{ + gchar *endptr; + glong val; + + errno = 0; + val = strtoll(s, &endptr, 10); + + if ((errno == ERANGE && (val == LONG_MAX || val == LONG_MIN)) + || (errno != 0 && val == 0)) + return FALSE; + + if (endptr == s || *endptr != '\0') + return FALSE; + + *d = val; + return TRUE; +} + +static void +tf_substr(LogMessage *msg, gint argc, GString *argv[], GString *result) +{ + glong start, len; + + if (argc != 3) + return; + + if (!tf_substr_parse_int (argv[0]->str, &start)) + return; + if (!tf_substr_parse_int (argv[1]->str, &len)) + return; + + if (start > argv[2]->len || + start + len >= argv[2]->len) + return; + + g_string_append_len (result, argv[2]->str + start - 1, len); +} + +TEMPLATE_FUNCTION_SIMPLE(tf_substr); + typedef struct _TFCondState { FilterExprNode *filter; @@ -158,6 +203,7 @@ static Plugin basicfuncs_plugins[] = TEMPLATE_FUNCTION_PLUGIN(tf_echo, "echo"), TEMPLATE_FUNCTION_PLUGIN(tf_grep, "grep"), TEMPLATE_FUNCTION_PLUGIN(tf_if, "if"), + TEMPLATE_FUNCTION_PLUGIN(tf_substr, "substr"), }; gboolean -- 1.7.2.5

3 8

SSL certificate verification
by Peter Eckel 30 Apr '11

30 Apr '11

Hi all, I am currently setting up a remote logging facility for a customer who requires encryption and mutual authentication. After having run Syslog NG over Stunnel for several years, I am looking into the option to do the same withot stunnel, by using Syslog NG with the TLS option. My current version is 3.2.2, installed from the binary provided on the Syslog NG web site, on CentOS 5.5. My setup on the server side (stripped to the minimum): source s_remote { tcp( ip( 192.168.42.184 ) port( 601 ) tls( key_file("/opt/syslog-ng/etc/ssl/server_key.pem") cert_file("/opt/syslog-ng/etc/ssl/server_crt.pem") ca_dir( "/opt/syslog-ng/etc/ssl" ) peer_verify( required-trusted ) ) ); }; destination d_remote { file( "/var/log/remote" owner(root) group(root) perm(0600) ); }; log { source(s_remote); destination(d_remote); }; On the client side: source s_local { internal(); unix-stream( "/dev/log" ); file( "/proc/kmsg" ); }; destination d_remote { tcp( "192.168.42.184" port( 601 ) tls( key_file("/opt/syslog-ng/etc/ssl/client1_key.pem") cert_file("/opt/syslog-ng/etc/ssl/client1_crt.pem") ca_dir( "/opt/syslog-ng/etc/ssl" ) ) ); }; log { source(s_local); destination(d_remote); }; With the appropriate and valid client cert, it works perfrctly: [root@server1 ssl]# openssl x509 -noout -text < client1_crt.pem Certificate: Data: Version: 1 (0x0) Serial Number: 2 (0x2) Signature Algorithm: sha256WithRSAEncryption Issuer: O=xxx, OU=Client 1/emailAddress=xxx, L=xxx, ST=xxx, C=xx, CN=xxx Validity Not Before: Mar 8 15:53:24 2011 GMT Not After : Mar 7 15:53:24 2014 GMT Subject: C=xx, ST=xx, O=xx, OU=xx, CN=client1 Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (2048 bit) ... However, I copied over the cert of another client machine to client1, reconfigure Syslog NG to use it, and it works as well, though IMHO it shouldn't: [root@server1 ssl]# openssl x509 -noout -text < client2_crt.pem Certificate: Data: Version: 1 (0x0) Serial Number: 3 (0x3) Signature Algorithm: sha256WithRSAEncryption Issuer: O=xxx, OU=Client 2/emailAddress=xxx, L=xxx, ST=xxx, C=xx, CN=xxx Validity Not Before: Mar 8 15:54:35 2011 GMT Not After : Mar 7 15:54:35 2014 GMT Subject: C=xx, ST=xxx, O=xxx, OU=Client 2, CN=client2 Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (2048 bit) Modulus (2048 bit): ... Swapping certificates (client2's instead of server's in this case) on the server breaks trust (as expected): Mar 8 18:19:45 client1 syslog-ng[2991]: Syslog connection established; fd='10', server='AF_INET(192.168.42.184:601)', local='AF_INET(0.0.0.0:0)' Mar 8 18:20:13 client1 syslog-ng[2991]: Certificate subject does not match configured hostname; hostname='server', certificate='client2' Mar 8 18:20:13 client1 syslog-ng[2991]: SSL error while writing stream; tls_error='SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed' Mar 8 18:20:13 client1 syslog-ng[2991]: I/O error occurred while writing; fd='10', error='Broken pipe (32)' Doing the same on the client, however, does not. The Syslog NG server does not seem to validate the CN. Is this expected behaviour or a bug? Best regards, Peter.

4 10

filtering per device
by vijay 30 Apr '11

30 Apr '11

Hi All, I have 4 cisco devices having ip addresses for example 10.1.1.1 10.1.1.2 10.1.1.3 10.1.1.4 already i have filtering option such that only above informational level(level 6) will be captured.I have new requirement that to capture informational logs also of 10.1.1.4 not other devices.How to do that. thanks vijay

3 2

[PATCH] afmongodb: Make the collection name a simple string.
by Gergely Nagy 30 Apr '11

30 Apr '11

Instead of allowing templates in the collection name, make it a simple string instead, to allow for easily implementing bulk inserts in the future. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afmongodb/afmongodb.c | 30 ++++++++++-------------------- 1 files changed, 10 insertions(+), 20 deletions(-) diff --git a/modules/afmongodb/afmongodb.c b/modules/afmongodb/afmongodb.c index 284066a..929d35b 100644 --- a/modules/afmongodb/afmongodb.c +++ b/modules/afmongodb/afmongodb.c @@ -47,7 +47,7 @@ typedef struct /* Shared between main/writer; only read by the writer, never written */ gchar *db; - LogTemplate *coll; + gchar *coll; gchar *host; gint port; @@ -84,11 +84,9 @@ typedef struct mongo_connection *conn; gint32 seq_num; - GString *current_namespace; - gint ns_prefix_len; + gchar *ns; GString *current_value; - bson *bson_sel, *bson_upd, *bson_set; } MongoDBDestDriver; @@ -145,8 +143,8 @@ afmongodb_dd_set_collection(LogDriver *d, const gchar *collection) { MongoDBDestDriver *self = (MongoDBDestDriver *)d; - log_template_unref(self->coll); - self->coll = log_template_new(log_pipe_get_config(&d->super), NULL, collection); + g_free(self->coll); + self->coll = g_strdup(collection); } void @@ -177,7 +175,7 @@ afmongodb_dd_format_stats_instance(MongoDBDestDriver *self) static gchar persist_name[1024]; g_snprintf(persist_name, sizeof(persist_name), - "mongodb,%s,%u,%s,%s", self->host, self->port, self->db, self->coll->template); + "mongodb,%s,%u,%s,%s", self->host, self->port, self->db, self->coll); return persist_name; } @@ -187,7 +185,7 @@ afmongodb_dd_format_persist_name(MongoDBDestDriver *self) static gchar persist_name[1024]; g_snprintf(persist_name, sizeof(persist_name), - "afmongodb(%s,%u,%s,%s)", self->host, self->port, self->db, self->coll->template); + "afmongodb(%s,%u,%s,%s)", self->host, self->port, self->db, self->coll); return persist_name; } @@ -274,10 +272,6 @@ afmongodb_worker_insert (MongoDBDestDriver *self) g_free (oid); bson_finish (self->bson_sel); - g_string_truncate(self->current_namespace, self->ns_prefix_len); - log_template_append_format(self->coll, msg, NULL, LTZ_LOCAL, - self->seq_num, NULL, self->current_namespace); - for (i = 0; i < self->num_fields; i++) { log_template_format(self->fields[i].value, msg, NULL, LTZ_SEND, @@ -292,7 +286,7 @@ afmongodb_worker_insert (MongoDBDestDriver *self) bson_append_document (self->bson_upd, "$set", self->bson_set); bson_finish (self->bson_upd); - p = mongo_wire_cmd_update (1, self->current_namespace->str, 1, + p = mongo_wire_cmd_update (1, self->ns, 1, self->bson_sel, self->bson_upd); if (!mongo_packet_send (self->conn, p)) @@ -335,12 +329,8 @@ afmongodb_worker_thread (gpointer arg) NULL); success = afmongodb_dd_connect(self, FALSE); - self->current_namespace = g_string_sized_new(64); - self->ns_prefix_len = strlen (self->db) + 1; - self->current_namespace = - g_string_append_c (g_string_assign (self->current_namespace, - self->db), '.'); + self->ns = g_string_concat (self->db, ".", self->coll, NULL); self->current_value = g_string_sized_new(256); @@ -383,7 +373,7 @@ afmongodb_worker_thread (gpointer arg) afmongodb_dd_disconnect(self); - g_string_free (self->current_namespace, TRUE); + g_free (self->ns); g_string_free (self->current_value, TRUE); bson_free (self->bson_sel); @@ -516,7 +506,7 @@ afmongodb_dd_free(LogPipe *d) g_free(self->fields); g_free(self->db); - log_template_unref(self->coll); + g_free(self->coll); g_free(self->user); g_free(self->password); g_free(self->host); -- 1.7.2.5

2 2

Formatting syslogs using syslog-ng
by Pramod Pillai 30 Apr '11

30 Apr '11

Hi I need to format the below syslog message. Apr 13 07:37:28 host-mgr dhcpd: [ID 702911 local7.error] DHCPDISCOVER to Apr 13 03:37:15 host-mgr 23 3 DHCPDISCOVER My destination config looks like this. destination sol_dest { file("/var/log/sol_syslog" perm(0644) template("$S_DATE $HOST $FACILITY_NUM $LEVEL_NUM $MSGONLY\n") template_escape(no) ); }; With the template mentioned above I am getting output as shown below. I need to remove " [ID 702911 local7.error] " . How to do it. Apr 13 03:37:15 mgr-t 23 3 [ID 702911 local7.error] DHCPDISCOVER I am running this on Solaris machine with syslong-ng 3.1.2 Regards Pramod

2 1