Hello, Thanks for the update and a possible workaround for this issue. I just made a small test (I do not have cisco device) based on your logs. I used syslog, cisco and app-parser(in a form of default-network-driver) and the result: PARSER=syslog, PROGRAM=%SYS-6-LOGGINGHOST_STARTSTOP, MSG=Logging to host blahblah PARSER=cisco, PROGRAM= MSG=%SYS-6-LOGGINGHOST_STARTSTOP: Logging to host blahblah PARSER=app-parser, PROGRAM= MSG=%SYS-6-LOGGINGHOST_STARTSTOP: Logging to host blahblah Maybe in the feature you could give a chance for the default-network-driver because it seems to handle the exact thing you aim to do (and more). Btw the UTC time works, because now syslog parser things that UTC: is a program name, and it seeks for the message. -- Kokan On Mon, Sep 24, 2018 at 4:27 PM Nik Ambrosch <nik@ambrosch.com> wrote:
Update - updating all of our (previously misbehaving) Cisco devices to log a time zone there have been no more incidents of syslog-ng parsing ${FULLHOST} as ":"
On Mon, Sep 17, 2018 at 3:40 PM Nik Ambrosch <nik@ambrosch.com> wrote:
Yes, hostname was parsed properly in 3.9. Three macros that I use changed after the upgrade:
${FULLHOST} - old: "myhost.com" - new: ":"
${PROGRAM} - old: "" - new: "%IP_VFR-4-FRAG_TABLE_OVERFLOW"
${MSG} old: "%IP_VFR-4-FRAG_TABLE_OVERFLOW: GigabitEthernet0/1: the fragment table..." new: "GigabitEthernet0/1: the fragment table..."
On Mon, Sep 17, 2018 at 3:26 PM Balazs Scheidler <bazsi77@gmail.com> wrote:
I think this would be handled by our cisco-parser() properly. Was this format properly parsed in 3.9?
On Mon, Sep 17, 2018 at 8:35 PM Nik Ambrosch <nik@ambrosch.com> wrote:
Confirmed when a cisco device does not send a timezone syslog-ng fails to properly parse the message. TImezone can be toggled on like this:
mydevice.com(config)#service timestamps log datetime show-timezone
On Mon, Sep 17, 2018 at 12:59 PM Nik Ambrosch <nik@ambrosch.com> wrote:
Well that's certainly easier, thanks :)
Patch did not succeed in fixing the issue - it may be because of the missing UTC that I mentioned in my earlier response?
Sep 11 12:14:51 1.1.1.1 <190>53: Sep 11 16:14:50.588: %SYS-6-LOGGINGHOST_STARTSTOP: broken device Sep 11 13:17:39 2.2.2.2 <190>10474: Sep 11 17:17:38.447 UTC: %SYS-6-LOGGINGHOST_STARTSTOP: working device
On Mon, Sep 17, 2018 at 12:36 PM Budai, László < laszlo.budai@oneidentity.com> wrote:
the PR has been merged six days ago, and this build has been finished four days ago: https://copr.fedorainfracloud.org/coprs/czanik/syslog-ng-githead/build/79769... which means, that these packages should contain that change.
L.
On Mon, Sep 17, 2018 at 6:29 PM, Nik Ambrosch <nik@ambrosch.com> wrote:
> Having a hard time getting lib/ivykis to compile on a centos 7.4 > system right now, haven't been able to figure it out yet. > > > On Mon, Sep 17, 2018 at 11:58 AM Budai, László < > laszlo.budai@oneidentity.com> wrote: > >> Hi, >> >> Do you need help in how to test the latest changes? >> >> L. >> >> >> On Mon, Sep 17, 2018 at 3:03 PM, Nik Ambrosch <nik@ambrosch.com> >> wrote: >> >>> Is there any other information I can provide in order to help >>> resolve this? >>> >>> >>> On Tue, Sep 11, 2018 at 1:48 PM Nik Ambrosch <nik@ambrosch.com> >>> wrote: >>> >>>> I setup a syslog-ng 3.9 device to capture a message using >>>> >>>> network(transport(tcp) flags(no-parse)); >>>> >>>> Here's what was logged: >>>> >>>> Sep 11 12:14:51 1.1.1.1 <190>53: Sep 11 16:14:50.588: >>>> %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host blahblah >>>> >>>> Followed up with a device that delivers a proper hostname, this >>>> is what was logged: >>>> >>>> Sep 11 13:17:39 2.2.2.2 <190>10474: Sep 11 17:17:38.447 UTC: >>>> %SYS-6-LOGGINGHOST_STARTSTOP: Logging to host blahblah >>>> >>>> Looks like the difference is the working device contains a >>>> timezone where as the non-working device does not. Everything else is the >>>> same however neither contain a hostname like in your example. >>>> >>>> >>>> >>>> >>>> >>>> On Tue, Sep 11, 2018 at 9:45 AM, Budai, László < >>>> laszlo.budai@oneidentity.com> wrote: >>>> >>>>> Hi, >>>>> >>>>> instead of reverting the ipv6 heuristic, I propose another >>>>> solution: >>>>> https://github.com/balabit/syslog-ng/pull/2272 >>>>> >>>>> I think that when a timestamp is followed by a colon(':'), it is >>>>> part of the timestamp and the (legacy) timestamp parser should 'eat' it. >>>>> >>>>> I tested with the following log: >>>>> <0>91: *Oct 07 03:10:04: mydevice.com %CRYPTO-4-RECVD_PKT_INV_SPI: >>>>> decaps: rec'd IPSEC packet has invalid spi for destaddr=150.1.1.1, prot=50, >>>>> spi=0x72662541(1919296833), srcaddr=150.3.1.3 >>>>> >>>>> Could you validate that this is the same format that you have? >>>>> >>>>> L. >>>>> >>>>> On Mon, Sep 10, 2018 at 4:32 PM, Scheidler, Balázs < >>>>> balazs.scheidler@oneidentity.com> wrote: >>>>> >>>>>> This branch has a patch to revert that specific commit, and >>>>>> I've confirmed that it resolves the issue for me, in exchange for not >>>>>> supporting IPV6 addresses in the hostname field. >>>>>> >>>>>> >>>>>> On Mon, Sep 10, 2018 at 3:55 PM, Balazs Scheidler < >>>>>> bazsi77@gmail.com> wrote: >>>>>> >>>>>>> This patch broke it: >>>>>>> >>>>>>> 399d565e9857e7cb41253e9a714d5cc6ad4d50fb. >>>>>>> >>>>>>> This patch can be reverted easily even on the latest master to >>>>>>> resolve the issue. >>>>>>> >>>>>>> On Mon, Sep 10, 2018 at 3:16 PM Scheidler, Balázs < >>>>>>> balazs.scheidler@oneidentity.com> wrote: >>>>>>> >>>>>>>> This is probably not it, the syslog-parser() changed some >>>>>>>> behaviours that changed it. >>>>>>>> >>>>>>>> On Mon, Sep 10, 2018, 13:45 Budai, László < >>>>>>>> laszlo.budai@oneidentity.com> wrote: >>>>>>>> >>>>>>>>> Hi, >>>>>>>>> >>>>>>>>> in syslog-ng OSE 3.13 [1] we introduced a new feature, >>>>>>>>> called app-parser [2] and the default network network driver is using it. >>>>>>>>> Maybe that could cause your issue. If this is the case, >>>>>>>>> then we have another PR [3] which makes it possible to disable the >>>>>>>>> auto-parse (also part of 3.13). >>>>>>>>> >>>>>>>>> Example: >>>>>>>>> source s_network { >>>>>>>>> default-network-drivers(auto-parse(no)); >>>>>>>>> }; >>>>>>>>> >>>>>>>>> If it not solves your problem then could you share the >>>>>>>>> relevant part of your config? >>>>>>>>> >>>>>>>>> >>>>>>>>> [1] >>>>>>>>> https://github.com/balabit/syslog-ng/releases/tag/syslog-ng-3.13.1 >>>>>>>>> [2] https://github.com/balabit/syslog-ng/pull/1689 >>>>>>>>> [3] https://github.com/balabit/syslog-ng/pull/1788/ >>>>>>>>> >>>>>>>>> >>>>>>>>> regards, >>>>>>>>> Laszlo Budai >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, Sep 7, 2018 at 6:00 PM, Nik Ambrosch < >>>>>>>>> nik@ambrosch.com> wrote: >>>>>>>>> >>>>>>>>>> Recently I upgraded my centralized loghost from 3.9 -> 3.15 >>>>>>>>>> and I noticed that some of my cisco devices started being logged in an >>>>>>>>>> undesirable format... I don't want to enable the cisco parser because more >>>>>>>>>> than just cisco messages get delivered to this interface. Here are the >>>>>>>>>> relevant fields that have changed before/after the upgrade: >>>>>>>>>> >>>>>>>>>> syslog-ng 3.9, before upgrade --- >>>>>>>>>> ${FULLHOST}: "mydevice.com" >>>>>>>>>> ${PROGRAM}: "" >>>>>>>>>> message: "%CRYPTO-4-RECVD_PKT_INV_SPI: decaps: rec'd >>>>>>>>>> IPSEC packet has invalid spi for..." >>>>>>>>>> >>>>>>>>>> syslog-ng 3.15, before upgrade --- >>>>>>>>>> ${FULLHOST}: ":" >>>>>>>>>> ${PROGRAM}: "%CRYPTO-4-RECVD_PKT_INV_SPI" >>>>>>>>>> ${MSG}: "decaps: rec'd IPSEC packet has invalid spi >>>>>>>>>> for..." >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Is this unintended behavior or a bug? This particular >>>>>>>>>> device is a Cisco 3845 running ios 12.4(22)T4. >>>>>>>>>> >>>>>>>>>> Thanks in advance. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> ______________________________________________________________________________ >>>>>>>>>> Member info: >>>>>>>>>> https://lists.balabit.hu/mailman/listinfo/syslog-ng >>>>>>>>>> Documentation: >>>>>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng >>>>>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>>> ______________________________________________________________________________ >>>>>>>>> Member info: >>>>>>>>> https://lists.balabit.hu/mailman/listinfo/syslog-ng >>>>>>>>> Documentation: >>>>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng >>>>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq >>>>>>>>> >>>>>>>>> >>>>>>>> ______________________________________________________________________________ >>>>>>>> Member info: >>>>>>>> https://lists.balabit.hu/mailman/listinfo/syslog-ng >>>>>>>> Documentation: >>>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng >>>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Bazsi >>>>>>> >>>>>>> >>>>>>> ______________________________________________________________________________ >>>>>>> Member info: >>>>>>> https://lists.balabit.hu/mailman/listinfo/syslog-ng >>>>>>> Documentation: >>>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng >>>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq >>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> ______________________________________________________________________________ >>>>>> Member info: >>>>>> https://lists.balabit.hu/mailman/listinfo/syslog-ng >>>>>> Documentation: >>>>>> http://www.balabit.com/support/documentation/?product=syslog-ng >>>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq >>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> ______________________________________________________________________________ >>>>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >>>>> Documentation: >>>>> http://www.balabit.com/support/documentation/?product=syslog-ng >>>>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq >>>>> >>>>> >>>>> >>>> >>> >>> ______________________________________________________________________________ >>> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >>> Documentation: >>> http://www.balabit.com/support/documentation/?product=syslog-ng >>> FAQ: http://www.balabit.com/wiki/syslog-ng-faq >>> >>> >>> >> >> ______________________________________________________________________________ >> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng >> Documentation: >> http://www.balabit.com/support/documentation/?product=syslog-ng >> FAQ: http://www.balabit.com/wiki/syslog-ng-faq >> >> > > ______________________________________________________________________________ > Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng > Documentation: > http://www.balabit.com/support/documentation/?product=syslog-ng > FAQ: http://www.balabit.com/wiki/syslog-ng-faq > > >
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
-- Bazsi
______________________________________________________________________________ Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq
Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng Documentation: http://www.balabit.com/support/documentation/?product=syslog-ng FAQ: http://www.balabit.com/wiki/syslog-ng-faq