[syslog-ng] Chained changing IP addresses to FQDN

[Lance Laursen]

On Sun, Dec 27, 2009 at 4:04 PM, Jakub Jankowski <shasta at toxcorp.com> wrote:

> On Monday 28 of December 2009, Christopher Barry wrote:
> > > >>
> > > >> I have a syslog infrastructure using syslog-ng comprising of three
> > > >> tiers: probes that accept messages and forwards them to the
> > > >> broker, a broker that routes and/or duplicates messages to the
> > > >> appropriate storage devices, and storage devices.  All forward
> > > >> using the tcp("hostname"); as a destination.
> [...]
> > > >> The probes rewrite the hostname to the IP address that the packet
> > > >> came from using the chain_hostnames(no) and keep_hostname(no)
> > > >> options.  One of the requirements of one specific storage device
> > > >> is to change the IP addresses into hostnames.  I'm having
> > > >> difficulty on the storage device switching the IP address in the
> > > >> log message to the FQDN.  I have tried all combinations I can
> > > >> think of of chain_hostnames and keep_hostname and cannot seem to
> > > >> get the IP address replaced with the FQDN.
> > > >>
> > > >> Is there a way to replace the IP address with the FQDN while
> > > >> receiving the messages from an intermediary?
> > > >
> > > > uhm, don't re-write it to ip addresses in the first place? Seems
> > > > logical, but I may not fully understand your implementation.
> > > >
> > > Our security team wants IP addresses from the packet for SIEM tools
> > > among other things and the sysadmins want FQDN instead of IP
> > > addresses since they know machines by hostname not IP.  Each team
> > > would have their own storage server.
> [...]
> > If it was me, I would see if I could do the splitting before the
> > conversion to all IP addresses. If this is simply not possible
> > architecturally, then I would look to see if syslog-ng had the ability
> > to route through a program as a filter (I'm betting it can, although I
> > can't say for sure.) If so, you could write a simple script to nslookup
> > or dig the IP address token in the stream, find the FQDN, and rewrite
> > it that way. If you do that, you'll definitely want the machine doing
> > the re-writing to have a good dnscache on it, so you're not going over
> > the wire all the time for lookups.
>
> See syslog-ng admin guide, look for use_dns(). You might find persist-only
> useful, combined with dns_cache_hosts().
>
> HTH
>
> --
> Jakub Jankowski|shasta at toxcorp.com|http://toxcorp.com/
> GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D
>
>
> ______________________________________________________________________________
> Member info: https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Documentation:
> http://www.balabit.com/support/documentation/?product=syslog-ng
> FAQ: http://www.campin.net/syslog-ng/faq.html
>
>
> ______________________________________________________________________
> This email has been scanned by the MessageLabs Email Security System.
> For more information please visit http://www.messagelabs.com/email
> ______________________________________________________________________
>

Hello,

Since you have a relay(broker) box in between the original log sender and
the storage device, you are going to run into a problem when you
use_dns(yes). Using the option keep_hostname(yes) simply tells syslog-ng to
set the $host to whatever it says in the second column of the log message it
is receiving. When you use keep_hostname(yes), use_dns() will be completely
ignored. When you set keep_hostname(no) and use_dns(yes), your logs are
simply going to be written out as coming from your probe or broker
(whichever relayed it to your storage device) because syslog will look at
the source IP of the packet it received the message from (broker) and then
do a reverse lookup on that in order to set the $host variable.

There are a couple of things you can do in your scenario. Your best bet may
be to recompile your syslog-ng instances with the --enable-spoof-source
option and use spoof_source(yes), and then use_dns(yes) at the storage box;
however, it only works with UDP, and i'm not sure of its behaviour over a
couple of hops of syslog boxes. Your next option is hacky, and that would be
to run your destinations to a script that does reverse lookups and rewrites
the host column. This option is rather terrifying to me, but I don't know
your log volume. I had thought there was somthing else you can do but I
can't seem to remember right now...I'll reply later if I do. In the meantime
grab the admin guide (
http://www.balabit.com/dl/guides/syslog-ng-v3.0-guide-admin-en.pdf ) and
look at the spoof-source option.

Keep in mind syslog-ng 3.x+ lets you use keep_hostname and use_dns on a
per-source basis, so you can set it for some sources but not for others.
Also, syslog-ng blocks on DNS lookups - if your dns goes down, your logs
(that have sources that use_dns) won't get written out. I don't know how
many hosts you have, so putting them in /etc/hosts then using the
use_dns(persist-only); dns_cache_hosts('/etc/hosts') options might be ok,
otherwise just set up a load of PTR records at your resolver (which is a
good idea anyways). There's also a use_fqdn option if your admins want that.

-- 
Lance Laursen
Demonware Systems Engineer
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.balabit.hu/pipermail/syslog-ng/attachments/20091227/8ddb99f4/attachment.htm