- syslog-ng - lists.balabit.hu

[PATCH] basicfuncs: Implement a $(substr START LEN STR) template function.
by Gergely Nagy 01 May '11

01 May '11

This implements the $(substr) template function - a fairly primitive one: it does error checking, but there's no error reporting present, therefore invalid function calls will just get ignored. Usage should be straightforward. Reported-By: Jakub Jankowski <shasta(a)toxcorp.com> Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/basicfuncs/basic-funcs.c | 46 ++++++++++++++++++++++++++++++++++++++ 1 files changed, 46 insertions(+), 0 deletions(-) diff --git a/modules/basicfuncs/basic-funcs.c b/modules/basicfuncs/basic-funcs.c index 44b8af3..542c2ec 100644 --- a/modules/basicfuncs/basic-funcs.c +++ b/modules/basicfuncs/basic-funcs.c @@ -4,6 +4,9 @@ #include "filter-expr-parser.h" #include "cfg.h" +#include <stdlib.h> +#include <errno.h> + static void tf_echo(LogMessage *msg, gint argc, GString *argv[], GString *result) { @@ -19,6 +22,48 @@ tf_echo(LogMessage *msg, gint argc, GString *argv[], GString *result) TEMPLATE_FUNCTION_SIMPLE(tf_echo); +static gboolean +tf_substr_parse_int(const gchar *s, long *d) +{ + gchar *endptr; + glong val; + + errno = 0; + val = strtoll(s, &endptr, 10); + + if ((errno == ERANGE && (val == LONG_MAX || val == LONG_MIN)) + || (errno != 0 && val == 0)) + return FALSE; + + if (endptr == s || *endptr != '\0') + return FALSE; + + *d = val; + return TRUE; +} + +static void +tf_substr(LogMessage *msg, gint argc, GString *argv[], GString *result) +{ + glong start, len; + + if (argc != 3) + return; + + if (!tf_substr_parse_int (argv[0]->str, &start)) + return; + if (!tf_substr_parse_int (argv[1]->str, &len)) + return; + + if (start > argv[2]->len || + start + len >= argv[2]->len) + return; + + g_string_append_len (result, argv[2]->str + start - 1, len); +} + +TEMPLATE_FUNCTION_SIMPLE(tf_substr); + typedef struct _TFCondState { FilterExprNode *filter; @@ -158,6 +203,7 @@ static Plugin basicfuncs_plugins[] = TEMPLATE_FUNCTION_PLUGIN(tf_echo, "echo"), TEMPLATE_FUNCTION_PLUGIN(tf_grep, "grep"), TEMPLATE_FUNCTION_PLUGIN(tf_if, "if"), + TEMPLATE_FUNCTION_PLUGIN(tf_substr, "substr"), }; gboolean -- 1.7.2.5

3 8

SSL certificate verification
by Peter Eckel 01 May '11

01 May '11

Hi all, I am currently setting up a remote logging facility for a customer who requires encryption and mutual authentication. After having run Syslog NG over Stunnel for several years, I am looking into the option to do the same withot stunnel, by using Syslog NG with the TLS option. My current version is 3.2.2, installed from the binary provided on the Syslog NG web site, on CentOS 5.5. My setup on the server side (stripped to the minimum): source s_remote { tcp( ip( 192.168.42.184 ) port( 601 ) tls( key_file("/opt/syslog-ng/etc/ssl/server_key.pem") cert_file("/opt/syslog-ng/etc/ssl/server_crt.pem") ca_dir( "/opt/syslog-ng/etc/ssl" ) peer_verify( required-trusted ) ) ); }; destination d_remote { file( "/var/log/remote" owner(root) group(root) perm(0600) ); }; log { source(s_remote); destination(d_remote); }; On the client side: source s_local { internal(); unix-stream( "/dev/log" ); file( "/proc/kmsg" ); }; destination d_remote { tcp( "192.168.42.184" port( 601 ) tls( key_file("/opt/syslog-ng/etc/ssl/client1_key.pem") cert_file("/opt/syslog-ng/etc/ssl/client1_crt.pem") ca_dir( "/opt/syslog-ng/etc/ssl" ) ) ); }; log { source(s_local); destination(d_remote); }; With the appropriate and valid client cert, it works perfrctly: [root@server1 ssl]# openssl x509 -noout -text < client1_crt.pem Certificate: Data: Version: 1 (0x0) Serial Number: 2 (0x2) Signature Algorithm: sha256WithRSAEncryption Issuer: O=xxx, OU=Client 1/emailAddress=xxx, L=xxx, ST=xxx, C=xx, CN=xxx Validity Not Before: Mar 8 15:53:24 2011 GMT Not After : Mar 7 15:53:24 2014 GMT Subject: C=xx, ST=xx, O=xx, OU=xx, CN=client1 Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (2048 bit) ... However, I copied over the cert of another client machine to client1, reconfigure Syslog NG to use it, and it works as well, though IMHO it shouldn't: [root@server1 ssl]# openssl x509 -noout -text < client2_crt.pem Certificate: Data: Version: 1 (0x0) Serial Number: 3 (0x3) Signature Algorithm: sha256WithRSAEncryption Issuer: O=xxx, OU=Client 2/emailAddress=xxx, L=xxx, ST=xxx, C=xx, CN=xxx Validity Not Before: Mar 8 15:54:35 2011 GMT Not After : Mar 7 15:54:35 2014 GMT Subject: C=xx, ST=xxx, O=xxx, OU=Client 2, CN=client2 Subject Public Key Info: Public Key Algorithm: rsaEncryption RSA Public Key: (2048 bit) Modulus (2048 bit): ... Swapping certificates (client2's instead of server's in this case) on the server breaks trust (as expected): Mar 8 18:19:45 client1 syslog-ng[2991]: Syslog connection established; fd='10', server='AF_INET(192.168.42.184:601)', local='AF_INET(0.0.0.0:0)' Mar 8 18:20:13 client1 syslog-ng[2991]: Certificate subject does not match configured hostname; hostname='server', certificate='client2' Mar 8 18:20:13 client1 syslog-ng[2991]: SSL error while writing stream; tls_error='SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed' Mar 8 18:20:13 client1 syslog-ng[2991]: I/O error occurred while writing; fd='10', error='Broken pipe (32)' Doing the same on the client, however, does not. The Syslog NG server does not seem to validate the CN. Is this expected behaviour or a bug? Best regards, Peter.

4 10

filtering per device
by vijay 30 Apr '11

30 Apr '11

Hi All, I have 4 cisco devices having ip addresses for example 10.1.1.1 10.1.1.2 10.1.1.3 10.1.1.4 already i have filtering option such that only above informational level(level 6) will be captured.I have new requirement that to capture informational logs also of 10.1.1.4 not other devices.How to do that. thanks vijay

3 2

[PATCH] afmongodb: Make the collection name a simple string.
by Gergely Nagy 30 Apr '11

30 Apr '11

Instead of allowing templates in the collection name, make it a simple string instead, to allow for easily implementing bulk inserts in the future. Signed-off-by: Gergely Nagy <algernon(a)balabit.hu> --- modules/afmongodb/afmongodb.c | 30 ++++++++++-------------------- 1 files changed, 10 insertions(+), 20 deletions(-) diff --git a/modules/afmongodb/afmongodb.c b/modules/afmongodb/afmongodb.c index 284066a..929d35b 100644 --- a/modules/afmongodb/afmongodb.c +++ b/modules/afmongodb/afmongodb.c @@ -47,7 +47,7 @@ typedef struct /* Shared between main/writer; only read by the writer, never written */ gchar *db; - LogTemplate *coll; + gchar *coll; gchar *host; gint port; @@ -84,11 +84,9 @@ typedef struct mongo_connection *conn; gint32 seq_num; - GString *current_namespace; - gint ns_prefix_len; + gchar *ns; GString *current_value; - bson *bson_sel, *bson_upd, *bson_set; } MongoDBDestDriver; @@ -145,8 +143,8 @@ afmongodb_dd_set_collection(LogDriver *d, const gchar *collection) { MongoDBDestDriver *self = (MongoDBDestDriver *)d; - log_template_unref(self->coll); - self->coll = log_template_new(log_pipe_get_config(&d->super), NULL, collection); + g_free(self->coll); + self->coll = g_strdup(collection); } void @@ -177,7 +175,7 @@ afmongodb_dd_format_stats_instance(MongoDBDestDriver *self) static gchar persist_name[1024]; g_snprintf(persist_name, sizeof(persist_name), - "mongodb,%s,%u,%s,%s", self->host, self->port, self->db, self->coll->template); + "mongodb,%s,%u,%s,%s", self->host, self->port, self->db, self->coll); return persist_name; } @@ -187,7 +185,7 @@ afmongodb_dd_format_persist_name(MongoDBDestDriver *self) static gchar persist_name[1024]; g_snprintf(persist_name, sizeof(persist_name), - "afmongodb(%s,%u,%s,%s)", self->host, self->port, self->db, self->coll->template); + "afmongodb(%s,%u,%s,%s)", self->host, self->port, self->db, self->coll); return persist_name; } @@ -274,10 +272,6 @@ afmongodb_worker_insert (MongoDBDestDriver *self) g_free (oid); bson_finish (self->bson_sel); - g_string_truncate(self->current_namespace, self->ns_prefix_len); - log_template_append_format(self->coll, msg, NULL, LTZ_LOCAL, - self->seq_num, NULL, self->current_namespace); - for (i = 0; i < self->num_fields; i++) { log_template_format(self->fields[i].value, msg, NULL, LTZ_SEND, @@ -292,7 +286,7 @@ afmongodb_worker_insert (MongoDBDestDriver *self) bson_append_document (self->bson_upd, "$set", self->bson_set); bson_finish (self->bson_upd); - p = mongo_wire_cmd_update (1, self->current_namespace->str, 1, + p = mongo_wire_cmd_update (1, self->ns, 1, self->bson_sel, self->bson_upd); if (!mongo_packet_send (self->conn, p)) @@ -335,12 +329,8 @@ afmongodb_worker_thread (gpointer arg) NULL); success = afmongodb_dd_connect(self, FALSE); - self->current_namespace = g_string_sized_new(64); - self->ns_prefix_len = strlen (self->db) + 1; - self->current_namespace = - g_string_append_c (g_string_assign (self->current_namespace, - self->db), '.'); + self->ns = g_string_concat (self->db, ".", self->coll, NULL); self->current_value = g_string_sized_new(256); @@ -383,7 +373,7 @@ afmongodb_worker_thread (gpointer arg) afmongodb_dd_disconnect(self); - g_string_free (self->current_namespace, TRUE); + g_free (self->ns); g_string_free (self->current_value, TRUE); bson_free (self->bson_sel); @@ -516,7 +506,7 @@ afmongodb_dd_free(LogPipe *d) g_free(self->fields); g_free(self->db); - log_template_unref(self->coll); + g_free(self->coll); g_free(self->user); g_free(self->password); g_free(self->host); -- 1.7.2.5

2 2

Formatting syslogs using syslog-ng
by Pramod Pillai 30 Apr '11

30 Apr '11

Hi I need to format the below syslog message. Apr 13 07:37:28 host-mgr dhcpd: [ID 702911 local7.error] DHCPDISCOVER to Apr 13 03:37:15 host-mgr 23 3 DHCPDISCOVER My destination config looks like this. destination sol_dest { file("/var/log/sol_syslog" perm(0644) template("$S_DATE $HOST $FACILITY_NUM $LEVEL_NUM $MSGONLY\n") template_escape(no) ); }; With the template mentioned above I am getting output as shown below. I need to remove " [ID 702911 local7.error] " . How to do it. Apr 13 03:37:15 mgr-t 23 3 [ID 702911 local7.error] DHCPDISCOVER I am running this on Solaris machine with syslong-ng 3.1.2 Regards Pramod

2 1

Not all messages getting to program destination
by Bill Anderson 30 Apr '11

30 Apr '11

Setup: v3.1 program destination: python script secondary file destination in the _same_ log block (so it should be getting everything the program destination gets), no final flag. stats turned on, 60s interval. Problem: In the script I am tallying lines parsed per minute. The lines parsed per-minute is not matching a grep -c on the file destination. stats indicate no dropped or queued messages to either destination. If I cat the file to the same destination script on the command line, I get the same counts as I do grepping the file directly. I would expect that the script would return the same results from the shell as when running as a destination in syslog. Where else can I look/what else can I turn on, to see why the discrepancy exists? Cheers, Bill -- Bill Anderson, RHCE Linux Systems Engineer bill.anderson(a)bodybuilding.com

4 7

Question about syntax for source
by Phusion 30 Apr '11

30 Apr '11

In syslog-ng 2.x versions under OpenBSD I used to use the following line for a source. source local { internal(); pipe("/dev/klog" log_prefix("kernel: ")); unix-dgram("/dev/log"); }; When trying to convert this for syslog-ng 3.x under OpenBSD. I have an error. When I use syslog-ng -s -f ../syslog-ng.conf it works, but when I run syslog-ng -p /var/run/syslog-ng.pid, I get the following error. # /usr/local/sbin/syslog-ng -p /var/run/syslog-ng.pid WARNING: you are using the pipe driver, underlying file is not a FIFO, it should be used by file(); filename='/dev/klog' Here is the current source line using syslog-ng 3.0 under OpenBSD 4.8. source local { internal(); pipe("/dev/klog" program_override("kernel: ")); unix-dgram("/dev/log"); }; Please advise. Phusion

6 12

log message difference between syslog and syslog-ng?
by Evan Baer 30 Apr '11

30 Apr '11

Hello, We switched one of our last remaining syslog servers to syslog-ng a few days ago, with no major surprises, except for one small difference in log message formatting between syslog and syslog-ng for messages from some of our network devices. The difference is in the added ':' in the 5th column, after the hostname: Apr 27 12:48:52 10.26.13.65 oob1-switch, System: Set fan speed to MED (75%) Apr 28 16:59:53 10.26.13.65 oob1-switch,: System: Set fan speed to LOW (50%) (the first line is from syslog, the second is from syslog-ng3-3.1.1, running on freebsd) For what it is worth, this formatting difference also occurs with other devices of the same manufacturer and model which have valid reverse ip entries in dns. We have a few support scripts which read the logs and are picky about the line formatting, so I'm keen to tweak the output lines in syslog-ng.conf if possible. Thoughts? Thanks in advance, -- Evan

2 1

syslog-ng and systemd
by Matthias Runge 30 Apr '11

30 Apr '11

-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hello users, Fedora 15 (and later) uses systemd as init-replacement. Currently, I'm building a fedora package for syslog-ng version 3.2.x. It is based on a git-snapshot dated April 24th. Is there a way to signal syslog-ng to reload it's configuration other than HUPing its pid? The documentation of systemd[1] says: ExecReload= should be specified for all services supporting reload. It is highly recommended to add code here that synchronously reloads the configuration file here (i.e. /bin/kill -HUP $MAINPID is usually a poor choice, due to its asynchronous nature). Omit this option if your service does not support reloading. I would highly appreciate not to maintain a pidfile. Newer linux kernels support so called cgroups. Those can be used to group processes. Systemd creates a cgroup for each service and starts in our case syslog-ng in this cgroup. When no process runs in syslog-ng cgroup, the process exited and must be restarted. systemd takes care of this. Again, is there any other way to signal syslog-ng to reload its config? Thanks, Matthias [1] https://fedoraproject.org/wiki/Systemd_Packaging_Draft#Writing_unit_files -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org/ iQEcBAEBAgAGBQJNuSg1AAoJEOnz8qQwcaIWb6oIAKMjeWwi5C/qZFeWgSL0/X5Y lsMjjWQ/YvhdgjHxNLEHFpTsJT6nrFo1akLN9nL4I8j23vekEgtDrWKAeOk/A+S9 LVUmgcGYnXJC3Oshu2Xg8MLcBQIIrXVrboTeG5GHGBG3gWmRuzL0yH9PjSKbNcd0 PtHSxqEBwhHN9vOc59a5UvclElGgmSzqmYdS5Qm84fStii+j7GNwkqRQvoEGrPlw RrOSFsIPGStXnxk+SzRn8JCKrk7ueZM7l8QoJPIfghdumFqyjPhZRd2BQUlorr6z J3u3GEpXaIOyj+qyix+KTTofjiJh/TC4/SVnqmNTQgsqKLNy1hDCWHXz2X1v7TY= =M50d -----END PGP SIGNATURE-----

3 2

gmane posting problem
by Micah Anderson 30 Apr '11

30 Apr '11

My previous message that I sent to this list I tried to send three different times via GMANE, which is subscribed to this list. Usually I have no issue posting to lists through this method, but for some reason, my message was not getting posted. Typically what happens is you post to the list via gmane, the gmane autoresponder sends you an email to confirm that you are a legit person and not a spammer, and then you reply and then you are allowed to post to the list. I did all that, but my messages were not coming through. Perhaps the list is configured in a strange way where these messages are getting rejected? thanks for looking into this, micah --

2 1