[syslog-ng] Rewrite engine and backreferences

[syslogng at feystorm.net]

Ya, csv is probably the better way to do that, but to answer your 
original question, it didnt work because you didnt tell it the 
replacement was a pcre regex (it defaults to posix standard regex) and 
posix regex doesnt understand most of the character classes pcre does 
(\S \s \d etc). Change it by specifying 'type(pcre)'.
You might also need to set the store-matches flag but I'm not sure on 
this. I know its required to access the captures from outside the match, 
but I havent tested to see if its required within a rewrite.

The administrators guide has all this in it and is a tremendously 
helpful resource. Can be found on the balabit web site under documentation.

-Patrick

Sent: Thursday, August 12, 2010 1:45:43 AM
From: Balint Kovacs <balint.kovacs at balabit.com>
To: syslog-ng at lists.balabit.hu
Subject: Re: [syslog-ng] Rewrite engine and backreferences
> Hi Mike,
>
> I think a less resource-intensive way of achieving this is to use a 
> csv-parser() and a custom template. You can tell the csv-parser() how 
> each column should be named and it saves the content of the field in a 
> variable, that you can use in the template later.
>
> So this would look something like:
>
> parser p_csv {
> csv-parser(columns(“APP.SYSTEM”, “APP.ACTION”, "APP.FLAGS", "APP.RESULT")
> delimiters(” “)
> template(“${MSG}”));
> };
>
> template t_csv {
> template(“$R_ISODATE $HOST $PROGRAM system=${APP.SYSTEM} 
> action=${APP.ACTION} flags=${APP.FLAGS} result=${APP.RESULT}\n");
> template-escape(no);
> };
>
> You can find more info on the csv-parser() here: 
> http://www.balabit.com/dl/html/syslog-ng-ose-v3.1-guide-admin-en.html/reference_parsers_csv.html
>
> Best Regards,
> Balint
>
> On 08/12/2010 08:19 AM, mike jack wrote:
>   
>> Hi list
>>
>> A system XYZ produces "csv-like" logs. Fields are separated by whitespace and different columns have different meanings. The column titles are not included in log messages.
>>
>> I would like to rewrite the log messages so that the column title is prefixed to each column contents. For example:
>>
>> qwerty asdfg zxcvb 12345
>>
>> would be rewritten to
>>
>> system=qwerty action=asdfg flags=zxcvb result=12345
>>
>> In perl that would be accomplished by backreferences, e.g.
>>
>> s/(\S+)\s+(\S+)\s+(\S+)\s+(\S+)/system=$1 action=$2 flags=$3 result=$4/
>>
>> This does not seem to work with syslog-ng rewrite rules out of the box. This is what I have tried:
>>
>> rewrite test_columns{subst("(\S+)\s+(\S+)\s+(\S+)\s+(\S+)", "system=$1 action=$2 flags=$3 result=$4", value("MESSAGE"));};
>>
>> The rewriting works without regexp, so I assume that the configuration syntax is correct.
>>
>> Is this possible with syslog-ng? If not using rewrite engine, does syslog-ng support feeding the logs through an external script? The system that produces the logs cannot be changed.
>>
>> The syslog-ng version is 3.0.4
>>
>> Thanks
>>
>> ....................................................................
>> Luukku Plus -paketilla pääset eroon tila- ja turvallisuusongelmista.
>> Hanki Luukku Plus ja helpotat elämääsi. http://www.mtv3.fi/luukku
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.campin.net/syslog-ng/faq.html
>>
>>
>>    
>>     
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>   
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20100812/e54bdce3/attachment.htm