[syslog-ng] syslog 3.9 modifying ${MSG}?

[Nik Ambrosch]

These are Cisco devices, I can reproduce with both old and new versions of ios.  Here is another message - i logged it twice, once with no-parse and one without that flag.

(using no flags):  %SYS-5-CONFIG_I: Configured from console by xxx on vty0 (0.0.0.0)
(no-parse flag):   <189>45: Apr 25 21:36:17.452 GMT: %SYS-5-CONFIG_I: Configured from console by xxx on vty0 (0.0.0.0)

Regarding the config & version - we moved from 3.3 to 3.9.  The behavior occurs when using sql() and program() destinations, i have not tried others but I would assume the same since it’s the value of a macro.

For reference, here is my old and new configuration for my database destination - note ${DEVICE_TYPE} is a custom macro.

old:

columns("id int(11) unsigned not null auto_increment primary key", "host varchar(100) not null", "facility varchar(10)", "priority varchar(10)", "level varchar(10)", "program text", "date date not null", "time time not null", "message text not null")
values("", "$FULLHOST", "$FACILITY", "$PRIORITY", "$LEVEL", "$PROGRAM", "$R_YEAR-$R_MONTH-$R_DAY", "$R_HOUR:$R_MIN:$R_SEC", "$MSG")


new:

columns("id int(11) unsigned not null auto_increment primary key", "host varchar(100) not null", "host_resolved varchar(60)", "facility varchar(10)", "priority varchar(10)", "level varchar(10)", "program varchar(60)", "date_local datetime not null", "date_remote datetime", "device_type varchar(13)", "message mediumblob not null")
values("0", "${FULLHOST}", "${FULLHOST_FROM}", "${FACILITY}", "${PRIORITY}", "${LEVEL}", "${PROGRAM}", "${R_YEAR}-${R_MONTH}-${R_DAY} ${R_HOUR}:{$R_MIN}:${R_SEC}", "${S_YEAR}-${S_MONTH}-${S_DAY} ${S_HOUR}:${S_MIN}:${S_SEC}", "${DEVICE_TYPE}", "${MSG}")

Thanks.




> On Apr 26, 2017, at 3:45 AM, Sandor Geller <sandor.geller at ericsson.com> wrote:
> 
> Hi,
> 
> The app producing these logs violates all syslog standards (there are so many apps written by people ignoring standards...). In my opinion syslog-ng is correct in assuming that the first doublecolon ends the syslog header and the string containing the doublecolon is the program name. I  don't know how could the older version mis-parse the message to pick up a not even existing string from the message as the program name.
> 
> Handling of $MSG AKA $MESSAGE changed with syslog-ng 3.0 and with recent configs (versioned ones having at least 3.0 in the version number) it no longer contains the syslog header. Were you using an old (unversioned) configfile and relied on 2.x behaviour maybe? syslog-ng outputs warnings about such behaviour changes when it starts.
> 
> The documentation also contains this information so it is a good read.
> 
> Regards,
> 
> Sandor
> 
> On 04/25/2017 09:23 PM, Nik Ambrosch wrote:
>> After moving from syslog-ng 3.5 to 3.9 i noticed that the contents of $PROGRAM and $MSG are being logged differently than before.  Here is how they used to be logged:
>> 
>> # program: 53
>> # message: Apr 19 09:35:35.713 GMT: %ILPOWER-5-POWER_GRANTED: Interface xxx: Power granted                                 |
>> 
>> That is the full message (as seen on the device) which is optimal behavior.  Below is the behavior i’m seeing with syslog-ng 3.9 with similar configuration:
>> 
>> # program: GMT
>> # message: %ILPOWER-5-POWER_GRANTED: Interface xxx: Power granted
>> 
>> The value of program and message are altered but everything else is the same.  I’ve been investigating the date parser and the flags(no-parse) options but haven’t had any luck getting a properly formatted message yet.
>> 
>> If anyone has any ideas on how to get the old behavior back it would be greatly appreciated.
>> 
>> ______________________________________________________________________________
>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
>> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>> 
> 
> 
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
>