[syslog-ng] spoof_source not working

[Jay Benfield]

Balazs,

I think we've figured it out. After doing a number of packet captures
and so forth we determined that the load balancer that the Solaris 9
syslog-ng servers sit behind are dropping outbound spoof_source
messages but are permitting messages that are not spoofed. Our Solaris
8 test server, that has always worked with spoof_source forwarding,
does not sit behind a load balancer.

This is consistent with the configuration of our load balancers and
was an oversight (read: duh) on our part. I think that once we get
this straightened out we'll be in good shape.

I appreciate everyone's help!

Thanks,
Jay

--
Japheth Benfield
http://benfield.org/

On 10/20/05, Balazs Scheidler <bazsi at balabit.hu> wrote:
> On Tue, 2005-10-18 at 14:28 -0400, Jay Benfield wrote:
>
> > For those new to this thread and wanting the Cliffs Notes, we have
> > been unable to get the 'spoof_source' feature working under Solaris 9.
> > In spite of syslog-ng being compiled with the --enable-spoof-source
> > option and the corresponding dependencies satisfied to the best of my
> > knowledge, when 'spoof_source(yes)' is present in a destination like:
> >
> >     destination d_foo { udp("<ip>" port(514) spoof_source(yes)); };
> >
> > with a corresponding log statement like:
> >
> >    log { source(net); destination(d_foo); };
> >
> > the destination syslog server does not receive a message at all. If
> > 'spoof_source(yes)' is removed from the configuration the syslog
> > message is received - sans source spoofing as expected.
> >
> > To make things even more interesting, we have a Solaris 8 server
> > running the same versions of software with the exact syslog-ng
> > configuration that works flawlessly. Both boxen are running the
> > following:
> >
> >     - syslog-ng 1.6.8
> >     - libol 0.3.16
> >     - libnet 1.1.2.1
> >
> > I captured the truss output of 'syslog-ng -dv' for a single message on
> > both the Solaris 8 server (operational) and the Solaris 9 server (soon
> > to become a boat anchor) with spoof_source enabled. I've made these
> > text files available at the following URL's:
> >
> >     http://benfield.org/lists/syslog-ng/truss_1message_sol8_20051018-01.txt
> >     http://benfield.org/lists/syslog-ng/truss_1message_sol9_20051018-01.txt
> >
> > I've also uploaded the truss output from the Solaris 9 server prior to
> > the syslog message being received.
> >
> >     http://benfield.org/lists/syslog-ng/truss_initial_sol9_20051018-01.txt
> >
> > Please let me know if any other information would be helpful. I
> > appreciate everyone's help!
>
> As I see from the truss output, the raw socket is opened (it becomes fd
> number 8), but no message is written to this fd afterwards for some
> reason.
>
> so_socket(PF_INET, SOCK_RAW, IPPROTO_RAW, "", 1) = 8
> setsockopt(8, ip, 2, 0xFFBFF864, 4, 1)          = 0
> getsockopt(8, SOL_SOCKET, SO_SNDBUF, 0xFFBFF864, 0xFFBFF860, 1) = 0
> setsockopt(8, SOL_SOCKET, SO_SNDBUF, 0xFFBFF864, 4, 1) = 0
>
> I can also see that instead of fd 8, syslog-ng writes the formatted UDP
> packet to fd 10, which is probably opened during initialization but was
> stripped from the truss output:
>
> setsockopt(8, SOL_SOCKET, SO_SNDBUF, 0xFFBFF864, 4, 1) = 0
> // MESSAGE ABOVE REPEATED ELEVENTY-BILLION TIMES
>
> setsockopt(10, SOL_SOCKET, SO_SNDBUF, 0xFFBFF864, 4, 1) Err#132 ENOBUFS
> setsockopt(10, SOL_SOCKET, SO_BROADCAST, 0xFFBFF864, 4, 1) = 0
>
> so syslog-ng definitely sends off a packet using fd 10, but I don't know
> what fd 10 is. Could you check your complete truss output to check
> whether it is there?
>
> Another question whether you one or two spoof_source enabled
> destination? As it seems you have at least two, because two similar fds
> are opened.
>
> Nevertheless syslog-ng (more precisely libnet) sends off the packet
> here:
>
> sendto(10, " E10\0 Z\0\0\0\0 @11 dED".., 90, 0, 0xFFBFF3C0, 16) = 90
>
> basically it is the same as on Solaris 8:
>
> sendto(8, " E10\0 Z\0\0\0\0 @11 dED".., 90, 0, 0xFFBEF2F8, 16) = 90
>
> (the only difference is the fd number used, but it does not really
> matter)
>
> My suspicion is that there's some packet filter/OS issue that prevents
> RAW sockets to work properly. Are you sure you cannot see the packet if
> you snoop the outgoing interface? Do you have proper outgoing routes?
>
> --
> Bazsi
>
> _______________________________________________
> syslog-ng maillist  -  syslog-ng at lists.balabit.hu
> https://lists.balabit.hu/mailman/listinfo/syslog-ng
> Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
>
>