<html><head><meta http-equiv="Content-Type" content="text/html charset=utf-8"></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;" class=""><div class="">Do you recall the version this was added in? I’m guessing 3.8?</div><div class=""><br class=""></div><div class="">There are non-cisco messages that arrive on this interface as well, hopefully there are no other conflicts. What do you think the best permanent solution is? Our centralized syslog server(s) accept messages from many different device types and i’m wondering if this will happen again in the future.</div><div class=""><br class=""></div><div class=""><br class=""></div><br class=""><div><blockquote type="cite" class=""><div class="">On Apr 26, 2017, at 10:07 AM, Scheidler, Balázs <<a href="mailto:balazs.scheidler@balabit.com" class="">balazs.scheidler@balabit.com</a>> wrote:</div><br class="Apple-interchange-newline"><div class=""><div dir="ltr" class=""><div class=""><div class="">Hi,<br class=""><br class=""></div><div class="">I think I've found the reason why it is parsed differently:<br class=""><br class=""></div><div class="">- we added support for parsing Cisco style sequence numbers (that's 45 in your example), so that does not get parsed as program name<br class=""></div><div class="">- however, we don't support timezones in the timestamps above, so GMT is being parsed as PROGRAM<br class=""></div><div class="">- adding support is not as simple as it sounds, as it conflicts with other log formats, as generally the host or program name is the next field, and those can be "GMT" as well.<br class=""><br class=""></div><div class="">This is what I would do:<br class=""><br class=""></div><div class="">- receive the messages using no-parse, then your entire message as received is available in $MSG<br class=""></div><div class="">- grab the timestamp with a regexp (e.g. message("<regexp>" flags(store-matches)))<br class=""></div><div class="">- use date-parser() to parse the timestamp<br class=""></div></div><br class=""></div><div class="gmail_extra"><br clear="all" class=""><div class=""><div class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr" class="">-- <br class="">Bazsi<br class=""></div></div></div>
<br class=""><div class="gmail_quote">On Wed, Apr 26, 2017 at 3:43 PM, Nik Ambrosch <span dir="ltr" class=""><<a href="mailto:nik@ambrosch.com" target="_blank" class="">nik@ambrosch.com</a>></span> wrote:<br class=""><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">These are Cisco devices, I can reproduce with both old and new versions of ios. Here is another message - i logged it twice, once with no-parse and one without that flag.<br class="">
<br class="">
(using no flags): %SYS-5-CONFIG_I: Configured from console by xxx on vty0 (0.0.0.0)<br class="">
(no-parse flag): <189>45: Apr 25 21:36:17.452 GMT: %SYS-5-CONFIG_I: Configured from console by xxx on vty0 (0.0.0.0)<br class="">
<br class="">
Regarding the config & version - we moved from 3.3 to 3.9. The behavior occurs when using sql() and program() destinations, i have not tried others but I would assume the same since it’s the value of a macro.<br class="">
<br class="">
For reference, here is my old and new configuration for my database destination - note ${DEVICE_TYPE} is a custom macro.<br class="">
<br class="">
old:<br class="">
<br class="">
columns("id int(11) unsigned not null auto_increment primary key", "host varchar(100) not null", "facility varchar(10)", "priority varchar(10)", "level varchar(10)", "program text", "date date not null", "time time not null", "message text not null")<br class="">
values("", "$FULLHOST", "$FACILITY", "$PRIORITY", "$LEVEL", "$PROGRAM", "$R_YEAR-$R_MONTH-$R_DAY", "$R_HOUR:$R_MIN:$R_SEC", "$MSG")<br class="">
<br class="">
<br class="">
new:<br class="">
<br class="">
columns("id int(11) unsigned not null auto_increment primary key", "host varchar(100) not null", "host_resolved varchar(60)", "facility varchar(10)", "priority varchar(10)", "level varchar(10)", "program varchar(60)", "date_local datetime not null", "date_remote datetime", "device_type varchar(13)", "message mediumblob not null")<br class="">
values("0", "${FULLHOST}", "${FULLHOST_FROM}", "${FACILITY}", "${PRIORITY}", "${LEVEL}", "${PROGRAM}", "${R_YEAR}-${R_MONTH}-${R_DAY} ${R_HOUR}:{$R_MIN}:${R_SEC}", "${S_YEAR}-${S_MONTH}-${S_DAY} ${S_HOUR}:${S_MIN}:${S_SEC}", "${DEVICE_TYPE}", "${MSG}")<br class="">
<br class="">
Thanks.<br class="">
<div class="HOEnZb"><div class="h5"><br class="">
<br class="">
<br class="">
<br class="">
> On Apr 26, 2017, at 3:45 AM, Sandor Geller <<a href="mailto:sandor.geller@ericsson.com" class="">sandor.geller@ericsson.com</a>> wrote:<br class="">
><br class="">
> Hi,<br class="">
><br class="">
> The app producing these logs violates all syslog standards (there are so many apps written by people ignoring standards...). In my opinion syslog-ng is correct in assuming that the first doublecolon ends the syslog header and the string containing the doublecolon is the program name. I don't know how could the older version mis-parse the message to pick up a not even existing string from the message as the program name.<br class="">
><br class="">
> Handling of $MSG AKA $MESSAGE changed with syslog-ng 3.0 and with recent configs (versioned ones having at least 3.0 in the version number) it no longer contains the syslog header. Were you using an old (unversioned) configfile and relied on 2.x behaviour maybe? syslog-ng outputs warnings about such behaviour changes when it starts.<br class="">
><br class="">
> The documentation also contains this information so it is a good read.<br class="">
><br class="">
> Regards,<br class="">
><br class="">
> Sandor<br class="">
><br class="">
> On 04/25/2017 09:23 PM, Nik Ambrosch wrote:<br class="">
>> After moving from syslog-ng 3.5 to 3.9 i noticed that the contents of $PROGRAM and $MSG are being logged differently than before. Here is how they used to be logged:<br class="">
>><br class="">
>> # program: 53<br class="">
>> # message: Apr 19 09:35:35.713 GMT: %ILPOWER-5-POWER_GRANTED: Interface xxx: Power granted |<br class="">
>><br class="">
>> That is the full message (as seen on the device) which is optimal behavior. Below is the behavior i’m seeing with syslog-ng 3.9 with similar configuration:<br class="">
>><br class="">
>> # program: GMT<br class="">
>> # message: %ILPOWER-5-POWER_GRANTED: Interface xxx: Power granted<br class="">
>><br class="">
>> The value of program and message are altered but everything else is the same. I’ve been investigating the date parser and the flags(no-parse) options but haven’t had any luck getting a properly formatted message yet.<br class="">
>><br class="">
>> If anyone has any ideas on how to get the old behavior back it would be greatly appreciated.<br class="">
>><br class="">
>> ______________________________<wbr class="">______________________________<wbr class="">__________________<br class="">
>> Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank" class="">https://lists.balabit.hu/<wbr class="">mailman/listinfo/syslog-ng</a><br class="">
>> Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank" class="">http://www.balabit.com/<wbr class="">support/documentation/?<wbr class="">product=syslog-ng</a><br class="">
>> FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank" class="">http://www.balabit.com/wiki/<wbr class="">syslog-ng-faq</a><br class="">
>><br class="">
><br class="">
><br class="">
> ______________________________<wbr class="">______________________________<wbr class="">__________________<br class="">
> Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank" class="">https://lists.balabit.hu/<wbr class="">mailman/listinfo/syslog-ng</a><br class="">
> Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank" class="">http://www.balabit.com/<wbr class="">support/documentation/?<wbr class="">product=syslog-ng</a><br class="">
> FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank" class="">http://www.balabit.com/wiki/<wbr class="">syslog-ng-faq</a><br class="">
><br class="">
<br class="">
______________________________<wbr class="">______________________________<wbr class="">__________________<br class="">
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" rel="noreferrer" target="_blank" class="">https://lists.balabit.hu/<wbr class="">mailman/listinfo/syslog-ng</a><br class="">
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" rel="noreferrer" target="_blank" class="">http://www.balabit.com/<wbr class="">support/documentation/?<wbr class="">product=syslog-ng</a><br class="">
FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" rel="noreferrer" target="_blank" class="">http://www.balabit.com/wiki/<wbr class="">syslog-ng-faq</a><br class="">
<br class="">
</div></div></blockquote></div><br class=""></div>
______________________________________________________________________________<br class="">Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" class="">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br class="">Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" class="">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br class="">FAQ: <a href="http://www.balabit.com/wiki/syslog-ng-faq" class="">http://www.balabit.com/wiki/syslog-ng-faq</a><br class=""><br class=""></div></blockquote></div><br class=""></body></html>