[syslog-ng] spoof_source not working

[Jay Benfield]

All,

Just to close out this thread, spoof_source is now working flawlessly
on our Solaris 9 boxen after adjusting our load balancer
configuration. Thanks again!

- Jay

--
Japheth Benfield
http://benfield.org/

On 10/20/05, Jay Benfield <jabenfield at gmail.com> wrote:
> Balazs,
>
> I think we've figured it out. After doing a number of packet captures
> and so forth we determined that the load balancer that the Solaris 9
> syslog-ng servers sit behind are dropping outbound spoof_source
> messages but are permitting messages that are not spoofed. Our Solaris
> 8 test server, that has always worked with spoof_source forwarding,
> does not sit behind a load balancer.
>
> This is consistent with the configuration of our load balancers and
> was an oversight (read: duh) on our part. I think that once we get
> this straightened out we'll be in good shape.
>
> I appreciate everyone's help!
>
> Thanks,
> Jay
>
> --
> Japheth Benfield
> http://benfield.org/
>
> On 10/20/05, Balazs Scheidler <bazsi at balabit.hu> wrote:
> > On Tue, 2005-10-18 at 14:28 -0400, Jay Benfield wrote:
> >
> > > For those new to this thread and wanting the Cliffs Notes, we have
> > > been unable to get the 'spoof_source' feature working under Solaris 9.
> > > In spite of syslog-ng being compiled with the --enable-spoof-source
> > > option and the corresponding dependencies satisfied to the best of my
> > > knowledge, when 'spoof_source(yes)' is present in a destination like:
> > >
> > >     destination d_foo { udp("<ip>" port(514) spoof_source(yes)); };
> > >
> > > with a corresponding log statement like:
> > >
> > >    log { source(net); destination(d_foo); };
> > >
> > > the destination syslog server does not receive a message at all. If
> > > 'spoof_source(yes)' is removed from the configuration the syslog
> > > message is received - sans source spoofing as expected.
> > >
> > > To make things even more interesting, we have a Solaris 8 server
> > > running the same versions of software with the exact syslog-ng
> > > configuration that works flawlessly. Both boxen are running the
> > > following:
> > >
> > >     - syslog-ng 1.6.8
> > >     - libol 0.3.16
> > >     - libnet 1.1.2.1
> > >
> > > I captured the truss output of 'syslog-ng -dv' for a single message on
> > > both the Solaris 8 server (operational) and the Solaris 9 server (soon
> > > to become a boat anchor) with spoof_source enabled. I've made these
> > > text files available at the following URL's:
> > >
> > >     http://benfield.org/lists/syslog-ng/truss_1message_sol8_20051018-01.txt
> > >     http://benfield.org/lists/syslog-ng/truss_1message_sol9_20051018-01.txt
> > >
> > > I've also uploaded the truss output from the Solaris 9 server prior to
> > > the syslog message being received.
> > >
> > >     http://benfield.org/lists/syslog-ng/truss_initial_sol9_20051018-01.txt
> > >
> > > Please let me know if any other information would be helpful. I
> > > appreciate everyone's help!
> >
> > As I see from the truss output, the raw socket is opened (it becomes fd
> > number 8), but no message is written to this fd afterwards for some
> > reason.
> >
> > so_socket(PF_INET, SOCK_RAW, IPPROTO_RAW, "", 1) = 8
> > setsockopt(8, ip, 2, 0xFFBFF864, 4, 1)          = 0
> > getsockopt(8, SOL_SOCKET, SO_SNDBUF, 0xFFBFF864, 0xFFBFF860, 1) = 0
> > setsockopt(8, SOL_SOCKET, SO_SNDBUF, 0xFFBFF864, 4, 1) = 0
> >
> > I can also see that instead of fd 8, syslog-ng writes the formatted UDP
> > packet to fd 10, which is probably opened during initialization but was
> > stripped from the truss output:
> >
> > setsockopt(8, SOL_SOCKET, SO_SNDBUF, 0xFFBFF864, 4, 1) = 0
> > // MESSAGE ABOVE REPEATED ELEVENTY-BILLION TIMES
> >
> > setsockopt(10, SOL_SOCKET, SO_SNDBUF, 0xFFBFF864, 4, 1) Err#132 ENOBUFS
> > setsockopt(10, SOL_SOCKET, SO_BROADCAST, 0xFFBFF864, 4, 1) = 0
> >
> > so syslog-ng definitely sends off a packet using fd 10, but I don't know
> > what fd 10 is. Could you check your complete truss output to check
> > whether it is there?
> >
> > Another question whether you one or two spoof_source enabled
> > destination? As it seems you have at least two, because two similar fds
> > are opened.
> >
> > Nevertheless syslog-ng (more precisely libnet) sends off the packet
> > here:
> >
> > sendto(10, " E10\0 Z\0\0\0\0 @11 dED".., 90, 0, 0xFFBFF3C0, 16) = 90
> >
> > basically it is the same as on Solaris 8:
> >
> > sendto(8, " E10\0 Z\0\0\0\0 @11 dED".., 90, 0, 0xFFBEF2F8, 16) = 90
> >
> > (the only difference is the fd number used, but it does not really
> > matter)
> >
> > My suspicion is that there's some packet filter/OS issue that prevents
> > RAW sockets to work properly. Are you sure you cannot see the packet if
> > you snoop the outgoing interface? Do you have proper outgoing routes?
> >
> > --
> > Bazsi
> >
> > _______________________________________________
> > syslog-ng maillist  -  syslog-ng at lists.balabit.hu
> > https://lists.balabit.hu/mailman/listinfo/syslog-ng
> > Frequently asked questions at http://www.campin.net/syslog-ng/faq.html
> >
> >
>