Ya, csv is probably the better way to do that, but to answer your original question, it didnt work because you didnt tell it the replacement was a pcre regex (it defaults to posix standard regex) and posix regex doesnt understand most of the character classes pcre does (\S \s \d etc). Change it by specifying 'type(pcre)'. You might also need to set the store-matches flag but I'm not sure on this. I know its required to access the captures from outside the match, but I havent tested to see if its required within a rewrite. The administrators guide has all this in it and is a tremendously helpful resource. Can be found on the balabit web site under documentation. -Patrick Sent: Thursday, August 12, 2010 1:45:43 AM From: Balint Kovacs <balint.kovacs@balabit.com> To: syslog-ng@lists.balabit.hu Subject: Re: [syslog-ng] Rewrite engine and backreferences

Hi Mike,

I think a less resource-intensive way of achieving this is to use a csv-parser() and a custom template. You can tell the csv-parser() how each column should be named and it saves the content of the field in a variable, that you can use in the template later.

So this would look something like:

parser p_csv { csv-parser(columns(“APP.SYSTEM”, “APP.ACTION”, "APP.FLAGS", "APP.RESULT") delimiters(” “) template(“${MSG}”)); };

template t_csv { template(“$R_ISODATE $HOST $PROGRAM system=${APP.SYSTEM} action=${APP.ACTION} flags=${APP.FLAGS} result=${APP.RESULT}\n"); template-escape(no); };

You can find more info on the csv-parser() here: http://www.balabit.com/dl/html/syslog-ng-ose-v3.1-guide-admin-en.html/refere...

Best Regards, Balint

On 08/12/2010 08:19 AM, mike jack wrote:

...

Hi list

A system XYZ produces "csv-like" logs. Fields are separated by whitespace and different columns have different meanings. The column titles are not included in log messages.

I would like to rewrite the log messages so that the column title is prefixed to each column contents. For example:

qwerty asdfg zxcvb 12345

would be rewritten to

system=qwerty action=asdfg flags=zxcvb result=12345

In perl that would be accomplished by backreferences, e.g.

s/(\S+)\s+(\S+)\s+(\S+)\s+(\S+)/system=$1 action=$2 flags=$3 result=$4/

This does not seem to work with syslog-ng rewrite rules out of the box. This is what I have tried:

rewrite test_columns{subst("(\S+)\s+(\S+)\s+(\S+)\s+(\S+)", "system=$1 action=$2 flags=$3 result=$4", value("MESSAGE"));};

The rewriting works without regexp, so I assume that the configuration syntax is correct.

Is this possible with syslog-ng? If not using rewrite engine, does syslog-ng support feeding the logs through an external script? The system that produces the logs cannot be changed.

The syslog-ng version is 3.0.4

Thanks

.................................................................... Luukku Plus -paketilla pääset eroon tila- ja turvallisuusongelmista. Hanki Luukku Plus ja helpotat elämääsi. http://www.mtv3.fi/luukku ______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html

______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.campin.net/syslog-ng/faq.html