[syslog-ng] Message and Header are being split incorrectly

[Sass, Fabian]

Hi Peter,

thanks a lot, this resolved the issue without having to use hacky templates.

Still it bugs me out that rewriting $PROGRAM didn't work. Any idea why?
According to the other list posting it should've.

Cheers,
Fabian

Von: syslog-ng <syslog-ng-bounces at lists.balabit.hu> Im Auftrag von Peter Czanik (pczanik)
Gesendet: Freitag, 5. Juni 2020 08:36
An: Syslog-ng users' and developers' mailing list <syslog-ng at lists.balabit.hu>
Betreff: Re: [syslog-ng] Message and Header are being split incorrectly

Hi,

FreeBSD 12.1 changed from the legacy syslog protocol to RFC 5424 format. When you use the system() source for local logs, this is handled automagically. But you can also fix it by hand: https://github.com/syslog-ng/syslog-ng/issues/2428

Bye,

Peter Czanik (CzP) <peter.czanik at oneidentity.com<mailto:peter.czanik at oneidentity.com>>
Balabit (a OneIdentity company) / syslog-ng upstream
https://syslog-ng.com/community/
https://twitter.com/PCzanik

________________________________
From: syslog-ng <syslog-ng-bounces at lists.balabit.hu<mailto:syslog-ng-bounces at lists.balabit.hu>> on behalf of Sass, Fabian <Fabian.Sass at f-i-ts.de<mailto:Fabian.Sass at f-i-ts.de>>
Sent: Friday, June 5, 2020 08:28
To: syslog-ng at lists.balabit.hu<mailto:syslog-ng at lists.balabit.hu> <syslog-ng at lists.balabit.hu<mailto:syslog-ng at lists.balabit.hu>>
Subject: [syslog-ng] Message and Header are being split incorrectly

CAUTION: This email originated from outside of the organization. Do not follow guidance, click links, or open attachments unless you recognize the sender and know the content is safe.


Hi,

since updating the Base OS to FreeBSD 12.1 syslog-ng 3.27.1 fails splitting a message and its headers.


Jun  5 08:12:00 myhostname 1 2020-06-05T08:12:00.042109+02:00 myhostname /usr/sbin/cron 71149 - - (root) RELOAD (tabs/root)

Jun  5 08:12:01 myhostname 1 2020-06-05T08:12:01.546089+02:00 myhostname named 54403 - - client @0xfffffff 0.0.0.0: update 'some.domain/IN' denied

Using templates for the destination reveals that the $MSGHDR Macro only holds the value "1", which is the wrongly extracted $PROGRAM macro.

$MESSAGE itself contains almost all information of the lines above, to be precise everything from (including) the $ISODATE to the end of the line.
However using templates and rewrite rules is in this case sufficient to restore the logformat that was used before the update.

The bigger issue is that changing the value of $PROGRAM has no effect when sending it to antoher syslog-ng loghost.
The behavior seems to be analog to this bug:
https://lists.balabit.hu/pipermail/syslog-ng/2011-August/017132.html<https://nam05.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fpipermail%2Fsyslog-ng%2F2011-August%2F017132.html&data=02%7C01%7CPeter.Czanik%40oneidentity.com%7C4be342ff90b940df59d008d80919b8e7%7C91c369b51c9e439c989c1867ec606603%7C0%7C1%7C637269353395272488&sdata=h4MEu8GH%2BKK6tbdxi8BvV6r5eBvoZ%2BF44JfzMLj6TbU%3D&reserved=0>

As you can see in my syslog-ng.conf the $PROGRAM macro is overwritten to "named" if named was logging to the local syslog-ng. The successful overwriting is verified using a separate logfile (destination d_test):



@version: 3.5



#

# options

#



options {

  mark_freq(3600);

  flush_lines(0);

  dir_perm(0640);

  chain_hostnames(off);

  keep_hostname(yes);

  create_dirs(yes);

  use_dns(yes);

  dns_cache(yes);

  dns_cache_expire(3600);

};
source s_all { unix-dgram("/var/run/log");

             unix-dgram("/var/run/logpriv" perm(0600));

         internal();

};



#

# rewrite since syslog message splitting is broken since update to freebsd12...

#

rewrite r_msg {

    set(

        "named", value("PROGRAM") condition(message(".* named [0-9]+ - -.*"))

    );

    subst(".* ([a-zA-Z/\._]+) ([0-9]+) - - (.*)", "$1[$2]: $3", value("MESSAGE"));

};



#

# destinations

#

destination d_test { file("/var/log/fabian_messages" template("$DATE $PROGRAM $HOST $MESSAGE\n")); };

destination d_test2 { file("/var/log/fabian_messages2"); };

destination d_messages { file("/var/log/messages" template("$DATE $HOST $MESSAGE\n")); };

destination d_loghost  { tcp("someiphere" port(514) template("$DATE $HOST $MESSAGE\n")); udp("anotheriphere" port(10525) template("$DATE $HOST $MESSAGE\n")); };

#



#

# logging

#

log { source(s_all); rewrite(r_msg); destination(d_messages); };

log { source(s_all); rewrite(r_msg); destination(d_loghost); };

log { source(s_all); rewrite(r_msg); destination(d_test); };

log { source(s_all); destination(d_test2); };



Am I missing something here, or is syslog-ng somehow behaving unintended?



Fabian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20200605/f7b10b3a/attachment.html>