Hi Mike, I think a less resource-intensive way of achieving this is to use a csv-parser() and a custom template. You can tell the csv-parser() how each column should be named and it saves the content of the field in a variable, that you can use in the template later. So this would look something like: parser p_csv { csv-parser(columns(“APP.SYSTEM”, “APP.ACTION”, "APP.FLAGS", "APP.RESULT") delimiters(” “) template(“${MSG}”)); }; template t_csv { template(“$R_ISODATE $HOST $PROGRAM system=${APP.SYSTEM} action=${APP.ACTION} flags=${APP.FLAGS} result=${APP.RESULT}\n"); template-escape(no); }; You can find more info on the csv-parser() here: http://www.balabit.com/dl/html/syslog-ng-ose-v3.1-guide-admin-en.html/refere... Best Regards, Balint On 08/12/2010 08:19 AM, mike jack wrote:
Hi list
A system XYZ produces "csv-like" logs. Fields are separated by whitespace and different columns have different meanings. The column titles are not included in log messages.
I would like to rewrite the log messages so that the column title is prefixed to each column contents. For example:
qwerty asdfg zxcvb 12345
would be rewritten to
system=qwerty action=asdfg flags=zxcvb result=12345
In perl that would be accomplished by backreferences, e.g.
s/(\S+)\s+(\S+)\s+(\S+)\s+(\S+)/system=$1 action=$2 flags=$3 result=$4/
This does not seem to work with syslog-ng rewrite rules out of the box. This is what I have tried:
rewrite test_columns{subst("(\S+)\s+(\S+)\s+(\S+)\s+(\S+)", "system=$1 action=$2 flags=$3 result=$4", value("MESSAGE"));};
The rewriting works without regexp, so I assume that the configuration syntax is correct.
Is this possible with syslog-ng? If not using rewrite engine, does syslog-ng support feeding the logs through an external script? The system that produces the logs cannot be changed.
The syslog-ng version is 3.0.4
Thanks
.................................................................... Luukku Plus -paketilla pääset eroon tila- ja turvallisuusongelmista. Hanki Luukku Plus ja helpotat elämääsi. http://www.mtv3.fi/luukku ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html