[syslog-ng] Colon embedded in messages (:)

[James Kelly]

Hello,

I am attempting to implement syslog-ng on our servers hosting postgresql
databases.  The general idea is to log "too much" at the database level and
then, using syslog filters, reduce it to the information we want to actually
hold in the syslog and send to the log server.  So far I am extremely
pleased with how easy it is to implement and well documented.  However, I do
have one problem that is making it difficult to use.

The problem is for each message that the filter matches, it does not
completely drop the message.. rather, it logs the date / server / and a
colon:

For example, this is what I get for a message that is matched by a filter:

"Sep 29 09:43:29 hcdb1-rep2      :"

I notice that even with the unmatched statements, there is a colon.  For
example, in the postgresql log, I see:

insert into "public"."table"(blah,blah,blah) values (blah,blah,blah);

but the same message once captured from syslog shows the following in the
log and logserver:

Sep 29 09:43:29 hcdb1-rep2      insert: into
"public"."table"(blah,blah,blah) values (blah,blah,blah);

*Note the colon after "insert".*  I can't seem to figure out where this is
coming from or how to avoid it.  It also causes some filtering problems that
I won't go into here so as to not confuse the issues, but safe to say it is
also related to the colon.

I have spent a lot of time trying to figure this out and am at a dead-end.
Due to the amount of messages I need to filter out (below is just one of the
many filters I need to put in and have tested with the same result), not
being able to completely filter these out is a killer.

I am using the 3.0.4 open-source edition on Ubuntu 8.  Here is my config:

******************************************************************************************
@version: 3.0

options {
};

######
# sources
source s_local {
# message generated by Syslog-NG
internal();
# standard Linux log source (this is the default place for the syslog()
# function to send logs to)
unix-stream("/dev/log");
# messages from the kernel
file("/proc/kmsg" program_override("kernel: "));
file("/var/log/postgresql/postgresql-8.3-main.log");
};


######
# destinations
destination d_messages { file("/var/log/messages"); };

#####
# filters

filter f_inserts2 {
        not match("_health_central" value("MESSAGE"));
};


destination d_logserver { tcp("internal.host.com"); };


log {
source(s_local);
filter(f_inserts2);
destination(d_messages);
#destination(d_logserver);
};
******************************************************************************************

Thanks!
James Kelly
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20090929/02575c25/attachment.htm