[tproxy] Need your input regarding some issue with tproxy (may not be related to tproxy)

[Pranav Desai]

On Thu, Oct 16, 2008 at 5:37 AM, Balazs Scheidler <bazsi at balabit.hu> wrote:
> On Wed, 2008-10-15 at 23:22 -0700, Pranav Desai wrote:
>> On Wed, Oct 15, 2008 at 5:29 PM, Pranav Desai <pranavadesai at gmail.com> wrote:
>> > Hello,
>> >
>> > We have a http proxy server which is in full transparent mode using
>> > tproxy 2.0.6 + kernel 2.6.20.15.
>> >
>> > The iptables rule to redirect port 80 traffic from clients is:
>> >
>> >  574K   34M REDIRECT   tcp  --  *      *       0.0.0.0/0
>> > 0.0.0.0/0           tcp dpts:80:82 redir ports 8001
>> >
>> > 8001 is our proxy port. We use tproxy to send connections to web
>> > servers using the clients IP. Using setsockopt(TPROXY_ASSIGN) ...
>> >
>> > The server is at around 240req/s. Most of the time its working fine
>> > i.e. I see a lot of traffic on port 80 going out using the clients IP,
>> > but I see quite a few pkts going out using src port 8001 (trace
>> > below). The rate is not as high port 80 traffic, but still high enough
>> > to be a concern.
>> >
>> > The interesting thing is I don't see any pkts destined to port 8001. I
>> > have been monitoring this server for an entire day. Hence, I am
>> > guessing that at times for some reason tproxy is not able to assign
>> > the foreign address and just uses the local address:port ...
>> >
>> > Any ideas or suggestions on how I should go about debugging this. Or
>> > whether I should be looking somewhere other than the tproxy module.
>> >
>> > Its a production server, so I can't get the client side traces, but I
>> > can get more info from the server. Let me know if you need any other
>> > information.
>> >
>> > I would really appreciate any help I can get.
>> >
>> > Thanks
>> > -- Pranav
>> >
>>
>>
>> The 10.10.224.6 is the server IP running tproxy.
>> The 10.1.x.x addresses are the client address.
>>
>> So the return traffic or the response seems to be having the problem
>> where sometimes the src is 10.10.224.6:8001 instead of the origin
>> server IP:PORT.
>
> Are you using bridging as well? Do you have CONFIG_NETFILTER_BRIDGE
> enabled in your kernel?
>

There is no bridging, only bonding.

> because in that case tcpdump will see the unnated traffic, bridging
> plays some nasty games with netfilter.
>
we thought that tcpdump might be screwing up somewhere, but even on
external network elements (load balancer) we don't see anything going
to the tproxy server on port 8001.

> On the client side of the proxy, the TPROXY_ASSIGN stuff does not really
> matter, only when going to the server.
>

Hmm. so does the netfilter nat rules control the client-side, and
could that be screwing up somehow. Can the conntrack table be of some
assistance here ?

Thanks

-- Pranav

> --
> Bazsi
>
>