All, Just to close out this thread, spoof_source is now working flawlessly on our Solaris 9 boxen after adjusting our load balancer configuration. Thanks again! - Jay -- Japheth Benfield http://benfield.org/ On 10/20/05, Jay Benfield <jabenfield@gmail.com> wrote:
Balazs,
I think we've figured it out. After doing a number of packet captures and so forth we determined that the load balancer that the Solaris 9 syslog-ng servers sit behind are dropping outbound spoof_source messages but are permitting messages that are not spoofed. Our Solaris 8 test server, that has always worked with spoof_source forwarding, does not sit behind a load balancer.
This is consistent with the configuration of our load balancers and was an oversight (read: duh) on our part. I think that once we get this straightened out we'll be in good shape.
I appreciate everyone's help!
Thanks, Jay
-- Japheth Benfield http://benfield.org/
On 10/20/05, Balazs Scheidler <bazsi@balabit.hu> wrote:
On Tue, 2005-10-18 at 14:28 -0400, Jay Benfield wrote:
For those new to this thread and wanting the Cliffs Notes, we have been unable to get the 'spoof_source' feature working under Solaris 9. In spite of syslog-ng being compiled with the --enable-spoof-source option and the corresponding dependencies satisfied to the best of my knowledge, when 'spoof_source(yes)' is present in a destination like:
destination d_foo { udp("<ip>" port(514) spoof_source(yes)); };
with a corresponding log statement like:
log { source(net); destination(d_foo); };
the destination syslog server does not receive a message at all. If 'spoof_source(yes)' is removed from the configuration the syslog message is received - sans source spoofing as expected.
To make things even more interesting, we have a Solaris 8 server running the same versions of software with the exact syslog-ng configuration that works flawlessly. Both boxen are running the following:
- syslog-ng 1.6.8 - libol 0.3.16 - libnet 1.1.2.1
I captured the truss output of 'syslog-ng -dv' for a single message on both the Solaris 8 server (operational) and the Solaris 9 server (soon to become a boat anchor) with spoof_source enabled. I've made these text files available at the following URL's:
http://benfield.org/lists/syslog-ng/truss_1message_sol8_20051018-01.txt http://benfield.org/lists/syslog-ng/truss_1message_sol9_20051018-01.txt
I've also uploaded the truss output from the Solaris 9 server prior to the syslog message being received.
http://benfield.org/lists/syslog-ng/truss_initial_sol9_20051018-01.txt
Please let me know if any other information would be helpful. I appreciate everyone's help!
As I see from the truss output, the raw socket is opened (it becomes fd number 8), but no message is written to this fd afterwards for some reason.
so_socket(PF_INET, SOCK_RAW, IPPROTO_RAW, "", 1) = 8 setsockopt(8, ip, 2, 0xFFBFF864, 4, 1) = 0 getsockopt(8, SOL_SOCKET, SO_SNDBUF, 0xFFBFF864, 0xFFBFF860, 1) = 0 setsockopt(8, SOL_SOCKET, SO_SNDBUF, 0xFFBFF864, 4, 1) = 0
I can also see that instead of fd 8, syslog-ng writes the formatted UDP packet to fd 10, which is probably opened during initialization but was stripped from the truss output:
setsockopt(8, SOL_SOCKET, SO_SNDBUF, 0xFFBFF864, 4, 1) = 0 // MESSAGE ABOVE REPEATED ELEVENTY-BILLION TIMES
setsockopt(10, SOL_SOCKET, SO_SNDBUF, 0xFFBFF864, 4, 1) Err#132 ENOBUFS setsockopt(10, SOL_SOCKET, SO_BROADCAST, 0xFFBFF864, 4, 1) = 0
so syslog-ng definitely sends off a packet using fd 10, but I don't know what fd 10 is. Could you check your complete truss output to check whether it is there?
Another question whether you one or two spoof_source enabled destination? As it seems you have at least two, because two similar fds are opened.
Nevertheless syslog-ng (more precisely libnet) sends off the packet here:
sendto(10, " E10\0 Z\0\0\0\0 @11 dED".., 90, 0, 0xFFBFF3C0, 16) = 90
basically it is the same as on Solaris 8:
sendto(8, " E10\0 Z\0\0\0\0 @11 dED".., 90, 0, 0xFFBEF2F8, 16) = 90
(the only difference is the fd number used, but it does not really matter)
My suspicion is that there's some packet filter/OS issue that prevents RAW sockets to work properly. Are you sure you cannot see the packet if you snoop the outgoing interface? Do you have proper outgoing routes?
-- Bazsi
_______________________________________________ syslog-ng maillist - syslog-ng@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/syslog-ng Frequently asked questions at http://www.campin.net/syslog-ng/faq.html