RE: [syslog-ng] Forwarding + Spoofing = Errors & Dropped Packets?
Thanks Bazsi. I finally got to the bottom of this, quite on accident. On a whim (since nothing else was working), I changed the destination for forwarding from hostnames to IP addresses. After I stopped and restarted the syslog-ng service, all worked well. I noticed in all my ltrace outputs that syslog-ng was performing a LOT of DNS queries when spoofing was on and we were using hostnames as the targets. There is not a local DNS server with this syslog-ng server, so the queries were taking quite a bit of time to come back (50-60 ms). Once I made the change to IP addresses, everything worked much better. I have now set my other syslog-ng server to use IP addresses instead of hostnames for forwarding as well. Whoever runs the syslog-ng FAQ on campin.net may need to know that if you have performance issues, switching your targets to IP addresses instead of hostnames may clear up some issues. Can you explain to me, though, why the spoofing needs to do so many nslookups? Why can it not cache the results of the first query? Does the application block waiting for DNS queries to come back? That may need to be addressed.... Thanks! Chris Ivey Affiliated Computer Services Enterprise Management Integration Services Infrastructure Management Senior Analyst 1120 Celebration Blvd. Celebration, FL 34747 (321) 939-6540 Office 8296-6540 Disney Tie-Line (407) 557-1072 Cell 162*326905*19 Nextel Direct Connect chris.ivey@acs-inc.com "When you find yourself in a hole, the best thing to do is stop digging!" -- Nick Stokes -----Original Message----- From: syslog-ng-bounces@lists.balabit.hu [mailto:syslog-ng-bounces@lists.balabit.hu] On Behalf Of Balazs Scheidler Sent: Sunday, January 14, 2007 9:47 AM To: Syslog-ng users' and developers' mailing list Subject: RE: [syslog-ng] Forwarding + Spoofing = Errors & Dropped Packets? On Thu, 2007-01-11 at 10:24 -0600, Ivey, Chris wrote:
Bazsi, Thanks for your help thus far. After MUCH trial and error with enabling and disabling things in syslog-ng.conf, and watching "netstat -an" output after each attempt, I have narrowed this issue down to something to do with spoofing. From my starting syslog-ng.conf file, if I do NOTHING but turn off spoofing, the Recv-Q buffer stops filling and all is happy. If I turn on spoofing, those buffers start filling again. I used tkdiff to compare my 2 syslog-ng.conf files side-by-side, and the only differences between the 2 are the lines where the forward-to hosts are defined and where the destinations are defined (the bad server has lines commented, the good one does not). Lsof on both machines for syslog-ng shows the same libraries opened and such. So, the question now is what happens with spoofing that would cause all these issues? What does spoofing do in the background that my server is unhappy about? Any insight would be greatly appreciated!
As the server is "unhappy", it is probably some kind of spoof filtering, e.g. the default route of the server points to a different interface, and the server drops the packets because of this. -- Bazsi _______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
On Mon, 2007-01-15 at 12:52 -0600, Ivey, Chris wrote:
Thanks Bazsi. I finally got to the bottom of this, quite on accident. On a whim (since nothing else was working), I changed the destination for forwarding from hostnames to IP addresses. After I stopped and restarted the syslog-ng service, all worked well. I noticed in all my ltrace outputs that syslog-ng was performing a LOT of DNS queries when spoofing was on and we were using hostnames as the targets. There is not a local DNS server with this syslog-ng server, so the queries were taking quite a bit of time to come back (50-60 ms). Once I made the change to IP addresses, everything worked much better. I have now set my other syslog-ng server to use IP addresses instead of hostnames for forwarding as well.
Whoever runs the syslog-ng FAQ on campin.net may need to know that if you have performance issues, switching your targets to IP addresses instead of hostnames may clear up some issues.
Can you explain to me, though, why the spoofing needs to do so many nslookups? Why can it not cache the results of the first query? Does the application block waiting for DNS queries to come back? That may need to be addressed.... Thanks!
Sure, this is a bug. When I add spoof-source support to 2.0.x, I'll take care of this. -- Bazsi
participants (2)
-
Balazs Scheidler
-
Ivey, Chris