[syslog-ng] hostname not appearing correctly when receiving logs from switches

Tue Jun 19 18:33:20 UTC 2018

No. cisco-parser() recognizes if a message is coming from cisco and parses
it properly if it does. The implementation is an scl macro that is
installed along syslog-ng, so you can easily tweak it to accept additional
formats.

The model with these high level macros is that you parse along a set of
conditionals:

log {
   source(whatever);
   if {
     cisco-parser ();
   } elif {
     apache-parser();
   };
};

The idea is that if cisco-parser() recognizes the message, apache-parser ()
never sees it.

A similar construct is implemented in the default-network-drivers() source
driver, which does two things:
* Opens all widely used ports
* Automatically parses incoming messages, recognizing their format with a
structure like above
* The automatic parsing is provided by the app-parser() framework

Everything is syslog-ng config language based, thus can be customized and
extended using the syslog-ng configuration language.

Bazsi

On Jun 19, 2018 19:22, "Clayton Dukes" <cdukes at logzilla.net> wrote:

> > cisco-parser()
>
> Doesn't the use of this flag require that **only** cisco events are being
> sent? What if someone is sending Juniper, Cisco, Linux, etc.?
>
>
>
>
>
>
>
>
>
> *From: *"Scheidler, Balázs" <balazs.scheidler at balabit.com>
> *Date: *Tuesday, June 19, 2018 at 7:58 AM
> *To: *Clayton Dukes <cdukes at logzilla.net>
> *Cc: *Syslog-ng users' and developers' mailing list <
> syslog-ng at lists.balabit.hu>
> *Subject: *Re: [syslog-ng] hostname not appearing correctly when
> receiving logs from switches
>
>
>
> Hi,
>
>
>
> I've finally got around and checked this out. Here's what I've found:
>
>
>
> * the message does not contain a hostname
>
> * if I am receiving this using a normal udp() source, I can reproduce that
> HOST becomes ':'
>
> * if I am using flags(no-parse) on input and the new cisco-parser() to
> actually decompose the message, it correctly parses it and HOST becomes the
> DNS resolved variant of the sender IP address.
>
>
>
> I'll check what 3.5 did in this scenario.
>
>
>
>
> --
> Bazsi
>
>
>
> On Mon, May 21, 2018 at 11:27 PM, Clayton Dukes <cdukes at logzilla.net>
> wrote:
>
> Was anyone able to figure out why this is happening?
>
>
>
> *From: *"Scheidler, Balázs" <balazs.scheidler at balabit.com>
> *Date: *Tuesday, May 15, 2018 at 10:58 AM
> *To: *"cdukes at cdukes.com" <cdukes at cdukes.com>
> *Cc: *Clayton Dukes <cdukes at logzilla.net>
> *Subject: *Re: Fwd: [syslog-ng] hostname not appearing correctly when
> receiving logs from switches
>
>
>
> Thanks
>
>
>
> On May 15, 2018 16:52, "Clayton Dukes" <cdukes at gmail.com> wrote:
>
> Making sure you got this (in case the other went to spam)
>
>
>
> ______________________________________________________________
>
> Clayton Dukes
> ______________________________________________________________
>
>
>
> ---------- Forwarded message ----------
> From: *Clayton Dukes* <cdukes at logzilla.net>
> Date: Mon, May 7, 2018 at 4:46 PM
> Subject: Re: [syslog-ng] hostname not appearing correctly when receiving
> logs from switches
> To: "Scheidler, Balázs" <balazs.scheidler at balabit.com>
> Cc: Syslog-ng users' and developers' mailing list <
> syslog-ng at lists.balabit.hu>
>
> Hi Balazs,
>
> Sorry for the delay, I don’t get a lot of free time these days :)
>
> I have attached a pcap as well as a raw log. The log is prior to any
> manipulation of LogZilla rules, etc.
>
>
>
> This is easily reproduceable.
>
> Also, if I add show-timezone to the device config, the host field shows
> up.
>
> The problem, of course, is that we can’t tell all of our customers to
> re-configure all of their cisco devices.
>
> We have documented the work-around here (search the page for “hostname
> missing”):
>
> http://demo.logzilla.net/help/receiving_data/cisco_ios_configuration
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fdemo.logzilla.net%2Fhelp%2Freceiving_data%2Fcisco_ios_configuration&data=02%7C01%7Ccdukes%40logzilla.net%7Cded1ab529711465a43c908d5d5dc090a%7C17fac5c255634489bef45cda2e65588f%7C0%7C0%7C636650063382360191&sdata=eu6dzttAGaSDJ38lwJBCW9ZrhLtcLW94a19%2BqhpWf%2BY%3D&reserved=0>
>
>
>
> This problem never existed before, but I am not certain which syslog-ng
> version it started occurring in.
>
>
>
>
>
>
>
>
>
> *From: *"Scheidler, Balázs" <balazs.scheidler at balabit.com>
> *Date: *Tuesday, May 1, 2018 at 8:45 PM
> *To: *Clayton Dukes <cdukes at logzilla.net>
> *Cc: *Joshua <aces621 at yahoo.com>, Syslog-ng users' and developers'
> mailing list <syslog-ng at lists.balabit.hu>
> *Subject: *Re: [syslog-ng] hostname not appearing correctly when
> receiving logs from switches
>
>
>
> Interesting that I saw this message the first time in your response, and
> not the original one.
>
>
>
> Anyhow, to understand the problem we would need an exact byte-by-byte
> representation of what syslog-ng is receiving from the switch together with
> the configuration that is used to process it. A tcpdump or an "Incoming
> message" from syslog debug outout should work.
>
>
>
> We haven't intentionally changed the syslog parser as far as I remember.
>
>
>
> On May 1, 2018 22:50, "Clayton Dukes" <cdukes at logzilla.net> wrote:
>
> Interesting! We’ve been getting a lot of support tickets for this very
> problem.
>
> I can easily recreate the issue.
>
>
>
> Balabit Team: is this a new bug?
>
>
>
>
>
>
>
> *[image: cid:image001.png at 01D306E3.0FEBC990]*
>
>
>
> *Clayton Dukes*
>
> Founder & CEO
>
> LogZilla Corporation
> 2900 N. Quinlan Park Rd
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fmaps.google.com%2F%3Fq%3D2900%2BN.%2BQuinlan%2BPark%2BRd%26entry%3Dgmail%26source%3Dg&data=02%7C01%7Ccdukes%40logzilla.net%7Cded1ab529711465a43c908d5d5dc090a%7C17fac5c255634489bef45cda2e65588f%7C0%7C0%7C636650063382360191&sdata=FrgwdfruMFVKpW6VfVvs%2F6n8zTCJ3J%2Bi54ksnS81yE8%3D&reserved=0>,
> B240-341
> Austin, TX, 78732
>
> Tel: 936-4NetOps (463-8677)
>
> Web: www.logzilla.net
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.logzilla.net%2F&data=02%7C01%7Ccdukes%40logzilla.net%7Cded1ab529711465a43c908d5d5dc090a%7C17fac5c255634489bef45cda2e65588f%7C0%7C0%7C636650063382360191&sdata=fUQ2yw9QScxbNLV%2Fd3AluolJNSIQt%2F%2B%2FrpT51HwQvOg%3D&reserved=0>
>
> [image: cid:image002.png at 01D306E3.0FEBC990]
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Ftwitter.com%2Flogzilla&data=02%7C01%7Ccdukes%40logzilla.net%7Cded1ab529711465a43c908d5d5dc090a%7C17fac5c255634489bef45cda2e65588f%7C0%7C0%7C636650063382360191&sdata=K%2FCa5OTPUVeDWkBE3S8stU%2Fr%2FlUgbvoDMSwk%2Byxg340%3D&reserved=0>[image:
> cid:image003.png at 01D306E3.0FEBC990]
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fyoutu.be%2Fdrg5wv_mgfA&data=02%7C01%7Ccdukes%40logzilla.net%7Cded1ab529711465a43c908d5d5dc090a%7C17fac5c255634489bef45cda2e65588f%7C0%7C0%7C636650063382360191&sdata=Zqm%2FuFkvDPLhIEWq5HLPNPFTeFZ%2BE6ez7Q0JEa7XDiA%3D&reserved=0>[image:
> cid:image004.png at 01D306E3.0FEBC990]
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.linkedin.com%2Fin%2Flzcdukes%2F&data=02%7C01%7Ccdukes%40logzilla.net%7Cded1ab529711465a43c908d5d5dc090a%7C17fac5c255634489bef45cda2e65588f%7C0%7C0%7C636650063382360191&sdata=Kr9AWm%2BRReWjoqZDkrtVE6J9C3gQ%2BrrZDR57m7naNAk%3D&reserved=0>
>
>
>
> *For NetOps, By NetOps!*
>
>
>
>
>
>
>
>
>
> *From: *syslog-ng <syslog-ng-bounces at lists.balabit.hu> on behalf of
> Joshua <aces621 at yahoo.com>
> *Reply-To: *Joshua <aces621 at yahoo.com>, Syslog-ng users' and developers'
> mailing list <syslog-ng at lists.balabit.hu>
> *Date: *Monday, April 30, 2018 at 7:09 PM
> *To: *"syslog-ng at lists.balabit.hu" <syslog-ng at lists.balabit.hu>
> *Subject: *[Suspected Spam] [syslog-ng] hostname not appearing correctly
> when receiving logs from switches
>
>
>
> Hi All,
>
>
>
> I am pretty new to syslog-ng but do have some basic knowledge. I have
> deployed syslog-ng v3.14 on a newly deployed Linux server because syslog-ng
> v3.5 is working very well on another syslog server.
>
>
>
> On this new deployment, the syslogs received from most of the servers are
> able to show IP/host, however, the syslogs from our switches contains
> IP/host showing as ":" (colons). I copied the current working custom build
> .conf from another syslog server into our new server. Can someone help me
> figure out what I am missing? It is working for some components but not for
> switches. I tested the same switch by sending syslog to another syslog
> server and the hostname is appearing but just not appearing on the new
> syslog server. The only difference between the two server is that one uses
> v3.5 (the working one) and the other uses syslog-ng v3.14.
>
>
>
> I have set: "keep_hostname (yes)" but it still doesn't work.
>
>
>
> Can someone please help? Am I missing something here?
>
>
>
> Thanks
>
> *Joshua Lai*
>
>
> ____________________________________________________________
> __________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> <https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.balabit.hu%2Fmailman%2Flistinfo%2Fsyslog-ng&data=02%7C01%7Ccdukes%40logzilla.net%7Cded1ab529711465a43c908d5d5dc090a%7C17fac5c255634489bef45cda2e65588f%7C0%7C0%7C636650063382360191&sdata=bOejwi1Ma3LTXLA6q8DyOCkg7U9d%2Baum11GZl2xHZGQ%3D&reserved=0>
> Documentation: http://www.balabit.com/support/documentation/?
> product=syslog-ng
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fsupport%2Fdocumentation%2F%3Fproduct%3Dsyslog-ng&data=02%7C01%7Ccdukes%40logzilla.net%7Cded1ab529711465a43c908d5d5dc090a%7C17fac5c255634489bef45cda2e65588f%7C0%7C0%7C636650063382360191&sdata=fjDO3a3fLjgecev1NQNh2IcncBpXJfFyo81dHJ8t0PY%3D&reserved=0>
> FAQ: http://www.balabit.com/wiki/syslog-ng-faq
> <https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.balabit.com%2Fwiki%2Fsyslog-ng-faq&data=02%7C01%7Ccdukes%40logzilla.net%7Cded1ab529711465a43c908d5d5dc090a%7C17fac5c255634489bef45cda2e65588f%7C0%7C0%7C636650063382360191&sdata=UD5jjaKFIwOWOu%2BOlDIfrLfhmJ25qa96gxDqu2v0F60%3D&reserved=0>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180619/f1b775b6/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image004.png
Type: image/png
Size: 1817 bytes
Desc: not available
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180619/f1b775b6/attachment-0004.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image001.png
Type: image/png
Size: 6538 bytes
Desc: not available
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180619/f1b775b6/attachment-0005.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image003.png
Type: image/png
Size: 1913 bytes
Desc: not available
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180619/f1b775b6/attachment-0006.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: image002.png
Type: image/png
Size: 2265 bytes
Desc: not available
URL: <http://lists.balabit.hu/pipermail/syslog-ng/attachments/20180619/f1b775b6/attachment-0007.png>