[syslog-ng] Rewrite regex not working entirely

[Nagy, Gábor]

Hi Max,

We have some guidelines for regexes, how to optimize them, syntax and
others:
https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.19/administration-guide/64#TOPIC-1094710
https://www.syslog-ng.com/technical-documents/doc/syslog-ng-open-source-edition/3.19/administration-guide/65

Regards,
Gabor

On Wed, Jan 16, 2019 at 5:33 PM N. Max Pierson <nmaxpierson at gmail.com>
wrote:

> Hi Atilla,
>
> I switched the double quotes to single quotes and that fixed the issue. I
> do not believe the docs stated to use single quotes for full regular
> expressions to work which is why I used double quotes but in either case
> this resolved the issue.
>
> Thanks for the feedback!
>
> Regards,
> Max
>
> On Wed, Jan 16, 2019 at 5:41 AM Szakacs, Attila <
> attila.szakacs at balabit.com> wrote:
>
>> Hi Max,
>>
>> I tried "\w\d" , etc... in "pcre" type subst rewrite rule on 3.19.
>> My config:
>>
>> @version: 3.19
>> @include "scl.conf"
>>
>> source s_udp5001 {
>>   udp(
>>     port(5001)
>>     keep-hostname(yes)
>>     flags(no-parse)
>>   );
>> };
>>
>> destination d_test {
>>   file(
>>     "/tmp/test.log"
>>   );
>> };
>>
>> rewrite r_chars {
>>   subst(
>>     "^[a-z]+ [0-9]+ [0-9]+:[0-9]+:[0-9]+ [a-z]+: ",
>>     "",
>>     value("$MESSAGE"),
>>     type("pcre"),
>>     flags("ignore-case")
>>   );
>> };
>>
>> rewrite r_pcre {
>>   subst(
>>     '^\w+\s\d+\s\d+:\d+:\d+\s\w+:\s',
>>     "",
>>     value("$MESSAGE"),
>>     type("pcre"),
>>     flags("ignore-case")
>>   );
>> };
>>
>> log {
>>   source(s_udp5001);
>>   #rewrite(r_chars);
>>   rewrite(r_pcre);
>>   destination(d_test);
>> };
>>
>> I think you need to make sure, that the regular expression is set between
>> single quotes (e.g.: '^\w+\s\d+\s\d+:\d+:\d+\s\w+:\s')
>>
>> Best regards,
>> Attila
>>
>> On Tue, Jan 15, 2019 at 11:12 PM N. Max Pierson <nmaxpierson at gmail.com>
>> wrote:
>>
>>> Hi Evan,
>>>
>>> I have tried both pcre and posix and neither seem to work.
>>>
>>> On Tue, Jan 15, 2019 at 4:08 PM Evan Rempel <erempel at uvic.ca> wrote:
>>>
>>>> You have defined your regular expresion as "posix" which does not have
>>>> the \d \s etc.
>>>> If you change the type to "pcre" then it should work for you.
>>>>
>>>>
>>>> On 1/15/19 2:01 PM, N. Max Pierson wrote:
>>>>
>>>> Hi List,
>>>>
>>>> I am using version 3.5 and it seems as though regex (posix or pcre)
>>>> doesn't work completely. Take the example string below (which is the
>>>> message part of the syslog).
>>>>
>>>> Jan 15 15:50:57 CST: %DAEMON-3-SYSTEM_MSG: NTP Receive dropping
>>>> message: Received NTP control mode packet. Drop count:147972  - ntpd[15029]
>>>>
>>>> I am trying to match the date at the beginning of the message and
>>>> remove it. When I use \w, \s, \d, etc, they do not match anything. If I
>>>> match on a character classes it works fine (ex [a-z]+ or [0-9]+).
>>>>
>>>> Here is my statement for the rewrite rule.
>>>>
>>>> rewrite r_nexus{ subst("^[a-z]+ [0-9]+ [0-9]+:[0-9]+:[0-9]+ [a-z]+: ",
>>>> "", value("MESSAGE"), type("posix"), flags("ignore-case"),
>>>> condition(filter(f_nexus))); };
>>>>
>>>> The above seems to get me what I want but are the character matches not
>>>> supposed to work in syslog-ng version 3.5??
>>>>
>>>> Regards,
>>>> Max
>>>>
>>>>
>>>>
>>>> ______________________________________________________________________________
>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>>> Documentation:
>>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>>
>>>>
>>> ______________________________________________________________________________
>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>>> Documentation:
>>> http://www.balabit.com/support/documentation/?product=syslog-ng
>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>>
>>>
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation:
>> http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>>
>>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20190117/6caf590f/attachment.html>