[syslog-ng] rewrite problem

[Balazs Scheidler]

On Tue, 2011-04-26 at 14:51 -0400, Micah Anderson wrote:
> Hi, 
> 
> I've been trying to use the rewrite functionality using 3.1.3, but I am
> having some problems:
> 
> 1. when having a rewrite setup, I can denial of service syslog-ng :(
> 
> if I do this:
> 
> rewrite r_ip { subst("([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}", "\[REDACTED\]", value("MESSAGE"), type("pcre"), flags("global")); };
> 
> filter f_mail { facility(mail) and not facility(local1) and not program("^postfix/"); };
> destination df_mail { file("/var/log/mail.log"); };
> 
> log {
>     source(s_src);
>     filter(f_mail);
>     rewrite(r_ip);
>     destination(df_mail);
>     flags(final);
> };
> 
> and then I do:
> 
> $ logger -p mail.info hi
> 
> syslog-ng starts going crazy, it eats 100% of the CPU and the memory. If
> you are lucky, the OOM killer kicks in and stops it before your system
> goes down.
> 
> I originally thought that this was due to a bad regexp I had, but the
> above regexp works fine for normal mail syslog messages. However, in the
> process of trying to make a better rewrite rule, I found that I've
> created ones where normal mail messages do this too.

Hmm... well, it is still probably a regexp problem, but it'd probably
make sense to validate it.

Hmm.. reading the code again, there's indeed a loop when "global"
replacements are made. However an infinite loop is only possible, when
the match has a zero length.

Can you please give me a log sample that causes syslog-ng to spin? I
couldn't reproduce it with a simplistic sample.

> 
> 2. \. doesn't match literal periods
> 
> Using the above rewrite, one would expect that the regexp would match
> three times a number containing 1-3 digits, each time followed by a
> literal period. Then a number containing 1-3 digits. A basic, and not
> very complete IPv4 regexp, that would match something like:
> 192.168.25.1.
> 
> In fact, it does match that IPv4 address, but it is also matching other
> things that do not have periods in them. For example:
> 
> rewrite r_ip { subst("([[:digit:]]{1,3}\.){3}[[:digit:]]{1,3}", "\[REDACTED\]", value("MESSAGE"), type("pcre"), flags("global")); };
> 
> $ logger -i -p mail.info 'Apr 21 20:38:58 algae postfix/smtp[14817]: EB4A2168: to=<482092-subscribe at bugs.debian.org>, relay=192.168.164.34[192.168.164.34]:587, delay=2.9, delays=0.19/0.14/2/0.58, dsn=2.0.0, status=sent (250 2.0.0 Ok: queued as 6722214C119)'
> 
> I would expect that the two IP addressesw in the above to be matched and
> replaced, but this is what happens:
> 
> Apr 22 12:45:11 algae logger[6234]: Apr [REDACTED] algae postfix/smtp[14817]: EB4A2168: to=<482092-subscribe at bugs.debian.org>, relay=[REDACTED][[REDACTED]]:587, delay=2.9, delays=[REDACTED]/2/0.58, dsn=2.0.0, status=sent ([REDACTED] Ok: queued as [REDACTED])
> 
> Its matching the queue ID 'EB4A2168' and its also matching
> '6722214C119', neither of those have periods in them, and EB4A2168 is
> not 0-9, which is what :digit: should be matching.
> 
> I tried to replace [[:digit:]] with \d, but that caused issue #1 to
> happen.

This is caused by the fact that syslog-ng strings (e.g. the ones
enclosed in double quotes) use \ for quotation too. In order to insert a
single backlslash into the regexp either use a double backslash or
single quotes, the latter which doesn't use quoting and inserts anything
enclosed in single quotes literally.

-- 
Bazsi