Thanks. This is exactly what is happening. If, using a template, I just use $PROGRAM, I get a bunch of lines with just "insert". <br><br>If I use something like the following:<br><br>template t_postgres_msgs {<br>
template("$ISODATE $HOST $PROGRAM $MSG \n"); };<br><br>I no longer see the colons. However, if something is matched by the filter, it still prints a blank message (hence, wasting a ton of space and clogging up the logs). For example, this is what I get instead of the matched sql statement:<br>
<br>2009-09-29T10:30:30-04:00 hcdb1-rep2<br><br>I thought the filter would just "trash" or not record the message at all, not print the host and timestamp for each filtered message.<br><br>Thanks a lot... I appreciate this help a lot.<br>
James<br><br><div class="gmail_quote">On Tue, Sep 29, 2009 at 10:05 AM, Srinivasan Sreenivasan <span dir="ltr"><<a href="mailto:srinivasan.srinivasan@sabre.com">srinivasan.srinivasan@sabre.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div>
<font size="4"><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size: 11pt;">I think Syslog-ng thinks insert is a program name. Use a template with $PROGRAM in it to see if it prints “insert” to confirm this.<br>
<br>
Solution:<br>
Send a program name before your sql statement using a template.<div><div></div><div class="h5"><br>
<br>
<br>
On 9/29/09 8:57 AM, "James Kelly" <<a href="mailto:james.kelly@hmsinc.com" target="_blank">james.kelly@hmsinc.com</a>> wrote:<br>
<br>
</div></div></span></font></font><blockquote><font size="4"><font face="Calibri, Verdana, Helvetica, Arial"><span style="font-size: 11pt;"><div><div></div><div class="h5">Hello,<br>
<br>
I am attempting to implement syslog-ng on our servers hosting postgresql databases. The general idea is to log "too much" at the database level and then, using syslog filters, reduce it to the information we want to actually hold in the syslog and send to the log server. So far I am extremely pleased with how easy it is to implement and well documented. However, I do have one problem that is making it difficult to use.<br>
<br>
The problem is for each message that the filter matches, it does not completely drop the message.. rather, it logs the date / server / and a colon:<br>
<br>
For example, this is what I get for a message that is matched by a filter:<br>
<br>
"Sep 29 09:43:29 hcdb1-rep2 :"<br>
<br>
I notice that even with the unmatched statements, there is a colon. For example, in the postgresql log, I see:<br>
<br>
insert into "public"."table"(blah,blah,blah) values (blah,blah,blah);<br>
<br>
but the same message once captured from syslog shows the following in the log and logserver:<br>
<br>
Sep 29 09:43:29 hcdb1-rep2 insert: into "public"."table"(blah,blah,blah) values (blah,blah,blah);<br>
<br>
*Note the colon after "insert".* I can't seem to figure out where this is coming from or how to avoid it. It also causes some filtering problems that I won't go into here so as to not confuse the issues, but safe to say it is also related to the colon.<br>
<br>
I have spent a lot of time trying to figure this out and am at a dead-end. Due to the amount of messages I need to filter out (below is just one of the many filters I need to put in and have tested with the same result), not being able to completely filter these out is a killer.<br>
<br>
I am using the 3.0.4 open-source edition on Ubuntu 8. Here is my config:<br>
<br>
******************************************************************************************<br>
@version: 3.0<br>
<br>
options {<br>
};<br>
<br>
######<br>
# sources<br>
source s_local {<br>
# message generated by Syslog-NG<br>
internal();<br>
# standard Linux log source (this is the default place for the syslog()<br>
# function to send logs to)<br>
unix-stream("/dev/log");<br>
# messages from the kernel<br>
file("/proc/kmsg" program_override("kernel: "));<br>
file("/var/log/postgresql/postgresql-8.3-main.log");<br>
};<br>
<br>
<br>
######<br>
# destinations<br>
destination d_messages { file("/var/log/messages"); };<br>
<br>
#####<br>
# filters<br>
<br>
filter f_inserts2 {<br>
not match("_health_central" value("MESSAGE"));<br>
};<br>
<br>
<br></div></div>
destination d_logserver { tcp("<a href="http://internal.host.com" target="_blank">internal.host.com</a> <a href="http://internal.host.com" target="_blank"><http://internal.host.com></a> "); };<div class="im">
<br>
<br>
<br>
log {<br>
source(s_local);<br>
filter(f_inserts2);<br>
destination(d_messages);<br>
#destination(d_logserver);<br>
};<br>
******************************************************************************************<br>
<br>
Thanks!<br>
James Kelly<br>
<br>
</div><hr width="95%" align="CENTER" size="3"></span></font><font face="Consolas, Courier New, Courier"><span style="font-size: 10pt;">______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.campin.net/syslog-ng/faq.html" target="_blank">http://www.campin.net/syslog-ng/faq.html</a><br>
<br>
</span></font></font></blockquote>
</div>
<br>______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.campin.net/syslog-ng/faq.html" target="_blank">http://www.campin.net/syslog-ng/faq.html</a><br>
<br>
<br></blockquote></div><br>