[syslog-ng] Rewrite engine and backreferences

[Balint Kovacs]

Hi Mike,

I think a less resource-intensive way of achieving this is to use a 
csv-parser() and a custom template. You can tell the csv-parser() how 
each column should be named and it saves the content of the field in a 
variable, that you can use in the template later.

So this would look something like:

parser p_csv {
csv-parser(columns(“APP.SYSTEM”, “APP.ACTION”, "APP.FLAGS", "APP.RESULT")
delimiters(” “)
template(“${MSG}”));
};

template t_csv {
template(“$R_ISODATE $HOST $PROGRAM system=${APP.SYSTEM} 
action=${APP.ACTION} flags=${APP.FLAGS} result=${APP.RESULT}\n");
template-escape(no);
};

You can find more info on the csv-parser() here: 
http://www.balabit.com/dl/html/syslog-ng-ose-v3.1-guide-admin-en.html/reference_parsers_csv.html

Best Regards,
Balint

On 08/12/2010 08:19 AM, mike jack wrote:
> Hi list
>
> A system XYZ produces "csv-like" logs. Fields are separated by whitespace and different columns have different meanings. The column titles are not included in log messages.
>
> I would like to rewrite the log messages so that the column title is prefixed to each column contents. For example:
>
> qwerty asdfg zxcvb 12345
>
> would be rewritten to
>
> system=qwerty action=asdfg flags=zxcvb result=12345
>
> In perl that would be accomplished by backreferences, e.g.
>
> s/(\S+)\s+(\S+)\s+(\S+)\s+(\S+)/system=$1 action=$2 flags=$3 result=$4/
>
> This does not seem to work with syslog-ng rewrite rules out of the box. This is what I have tried:
>
> rewrite test_columns{subst("(\S+)\s+(\S+)\s+(\S+)\s+(\S+)", "system=$1 action=$2 flags=$3 result=$4", value("MESSAGE"));};
>
> The rewriting works without regexp, so I assume that the configuration syntax is correct.
>
> Is this possible with syslog-ng? If not using rewrite engine, does syslog-ng support feeding the logs through an external script? The system that produces the logs cannot be changed.
>
> The syslog-ng version is 3.0.4
>
> Thanks
>
> ....................................................................
> Luukku Plus -paketilla pääset eroon tila- ja turvallisuusongelmista.
> Hanki Luukku Plus ja helpotat elämääsi. http://www.mtv3.fi/luukku
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>
>