<div class="gmail_quote">On Sun, Dec 27, 2009 at 4:04 PM, Jakub Jankowski <span dir="ltr"><<a href="mailto:shasta@toxcorp.com">shasta@toxcorp.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div class="im">On Monday 28 of December 2009, Christopher Barry wrote:<br>
> > >><br>
> > >> I have a syslog infrastructure using syslog-ng comprising of three<br>
> > >> tiers: probes that accept messages and forwards them to the<br>
> > >> broker, a broker that routes and/or duplicates messages to the<br>
> > >> appropriate storage devices, and storage devices. All forward<br>
> > >> using the tcp("hostname"); as a destination.<br>
</div>[...]<br>
<div class="im">> > >> The probes rewrite the hostname to the IP address that the packet<br>
> > >> came from using the chain_hostnames(no) and keep_hostname(no)<br>
> > >> options. One of the requirements of one specific storage device<br>
> > >> is to change the IP addresses into hostnames. I'm having<br>
> > >> difficulty on the storage device switching the IP address in the<br>
> > >> log message to the FQDN. I have tried all combinations I can<br>
> > >> think of of chain_hostnames and keep_hostname and cannot seem to<br>
> > >> get the IP address replaced with the FQDN.<br>
> > >><br>
> > >> Is there a way to replace the IP address with the FQDN while<br>
> > >> receiving the messages from an intermediary?<br>
> > ><br>
</div><div class="im">> > > uhm, don't re-write it to ip addresses in the first place? Seems<br>
> > > logical, but I may not fully understand your implementation.<br>
> > ><br>
</div><div class="im">> > Our security team wants IP addresses from the packet for SIEM tools<br>
> > among other things and the sysadmins want FQDN instead of IP<br>
> > addresses since they know machines by hostname not IP. Each team<br>
> > would have their own storage server.<br>
</div>[...]<br>
<div class="im">> If it was me, I would see if I could do the splitting before the<br>
> conversion to all IP addresses. If this is simply not possible<br>
> architecturally, then I would look to see if syslog-ng had the ability<br>
> to route through a program as a filter (I'm betting it can, although I<br>
> can't say for sure.) If so, you could write a simple script to nslookup<br>
> or dig the IP address token in the stream, find the FQDN, and rewrite<br>
> it that way. If you do that, you'll definitely want the machine doing<br>
> the re-writing to have a good dnscache on it, so you're not going over<br>
> the wire all the time for lookups.<br>
<br>
</div>See syslog-ng admin guide, look for use_dns(). You might find persist-only<br>
useful, combined with dns_cache_hosts().<br>
<br>
HTH<br>
<font color="#888888"><br>
--<br>
Jakub Jankowski|<a href="mailto:shasta@toxcorp.com">shasta@toxcorp.com</a>|<a href="http://toxcorp.com/" target="_blank">http://toxcorp.com/</a><br>
GPG: FCBF F03D 9ADB B768 8B92 BB52 0341 9037 A875 942D<br>
</font><div><div></div><div class="h5"><br>
______________________________________________________________________________<br>
Member info: <a href="https://lists.balabit.hu/mailman/listinfo/syslog-ng" target="_blank">https://lists.balabit.hu/mailman/listinfo/syslog-ng</a><br>
Documentation: <a href="http://www.balabit.com/support/documentation/?product=syslog-ng" target="_blank">http://www.balabit.com/support/documentation/?product=syslog-ng</a><br>
FAQ: <a href="http://www.campin.net/syslog-ng/faq.html" target="_blank">http://www.campin.net/syslog-ng/faq.html</a><br>
<br>
<br>
______________________________________________________________________<br>
This email has been scanned by the MessageLabs Email Security System.<br>
For more information please visit <a href="http://www.messagelabs.com/email" target="_blank">http://www.messagelabs.com/email</a><br>
______________________________________________________________________<br>
</div></div></blockquote></div><br>Hello,<br><br>Since you have a relay(broker) box in between the
original log sender and the storage device, you are going to run into a
problem when you use_dns(yes). Using the option keep_hostname(yes)
simply tells syslog-ng to set the $host to whatever it says in the
second column of the log message it is receiving. When you use
keep_hostname(yes), use_dns() will be completely ignored. When you set
keep_hostname(no) and use_dns(yes), your logs are simply going to be
written out as coming from your probe or broker (whichever relayed it
to your storage device) because syslog will look at the source IP of
the packet it received the message from (broker) and then do a reverse
lookup on that in order to set the $host variable.<br><br>There are a
couple of things you can do in your scenario. Your best bet may be to
recompile your syslog-ng instances with the --enable-spoof-source
option and use spoof_source(yes), and then use_dns(yes) at the storage
box; however, it only works with UDP, and i'm not sure of its behaviour
over a couple of hops of syslog boxes. Your next option is hacky, and
that would be to run your destinations to a script that does reverse
lookups and rewrites the host column. This option is rather terrifying
to me, but I don't know your log volume. I had thought there was
somthing else you can do but I can't seem to remember right now...I'll
reply later if I do. In the meantime grab the admin guide (
<a href="http://www.balabit.com/dl/guides/syslog-ng-v3.0-guide-admin-en.pdf">http://www.balabit.com/dl/guides/syslog-ng-v3.0-guide-admin-en.pdf</a> )
and look at the spoof-source option.<br><br>Keep in mind syslog-ng 3.x+
lets you use keep_hostname and use_dns on a per-source basis, so you
can set it for some sources but not for others. Also, syslog-ng blocks
on DNS lookups - if your dns goes down, your logs (that have sources
that use_dns) won't get written out. I don't know how many hosts you
have, so putting them in /etc/hosts then using the
use_dns(persist-only); dns_cache_hosts('/etc/hosts') options might be
ok, otherwise just set up a load of PTR records at your resolver (which
is a good idea anyways). There's also a use_fqdn option if your admins
want that.<br clear="all"><br>-- <br>Lance Laursen<br>Demonware Systems Engineer<br>