[tproxy] Need your input regarding some issue with tproxy (may not be related to tproxy)

[Pranav Desai]

Hello,

We have a http proxy server which is in full transparent mode using
tproxy 2.0.6 + kernel 2.6.20.15.

The iptables rule to redirect port 80 traffic from clients is:

 574K   34M REDIRECT   tcp  --  *      *       0.0.0.0/0
0.0.0.0/0           tcp dpts:80:82 redir ports 8001

8001 is our proxy port. We use tproxy to send connections to web
servers using the clients IP. Using setsockopt(TPROXY_ASSIGN) ...

The server is at around 240req/s. Most of the time its working fine
i.e. I see a lot of traffic on port 80 going out using the clients IP,
but I see quite a few pkts going out using src port 8001 (trace
below). The rate is not as high port 80 traffic, but still high enough
to be a concern.

The interesting thing is I don't see any pkts destined to port 8001. I
have been monitoring this server for an entire day. Hence, I am
guessing that at times for some reason tproxy is not able to assign
the foreign address and just uses the local address:port ...

Any ideas or suggestions on how I should go about debugging this. Or
whether I should be looking somewhere other than the tproxy module.

Its a production server, so I can't get the client side traces, but I
can get more info from the server. Let me know if you need any other
information.

I would really appreciate any help I can get.

Thanks
-- Pranav

Here is a small part of the trace using (tcpdump -vvv -nn -i bond1 port 8001)
===========================================================
03:15:19.915737 IP (tos 0x0, ttl  64, id 28888, offset 0, flags [DF],
proto 6, length: 40) 10.10.224.6.8001 > 10.1.1.8.4849: F [tcp sum ok]
4133498992:4133498992(0) ack 3495097709 win 7290
03:15:22.180160 IP (tos 0x0, ttl  64, id 12306, offset 0, flags [DF],
proto 6, length: 40) 10.10.224.6.8001 > 10.1.1.8.4826: F [tcp sum ok]
4117981558:4117981558(0) ack 2998309917 win 9310
03:15:24.578292 IP (tos 0x0, ttl  64, id 65515, offset 0, flags [DF],
proto 6, length: 40) 10.10.224.6.8001 > 10.1.2.47.1106: F [tcp sum ok]
4086941566:4086941566(0) ack 3480965117 win 6432
03:15:25.064570 IP (tos 0x0, ttl  64, id 38848, offset 0, flags [DF],
proto 6, length: 40) 10.10.224.6.8001 > 10.1.3.131.1044: F [tcp sum
ok] 4055165713:4055165713(0) ack 3713243767 win 9648
03:15:27.232857 IP (tos 0x0, ttl  64, id 17142, offset 0, flags [DF],
proto 6, length: 40) 10.10.224.6.8001 > 10.1.1.8.4888: F [tcp sum ok]
4134874226:4134874226(0) ack 2610855931 win 7380
03:15:30.079850 IP (tos 0x0, ttl  64, id 13364, offset 0, flags [DF],
proto 6, length: 1185) 10.10.224.6.8001 > 10.1.4.49.46316: P
22595600:22596733(1133) ack 805897755 win 1746 <nop,nop,timestamp
24258316 3379834444>
03:15:31.089784 IP (tos 0x0, ttl  64, id 37000, offset 0, flags [DF],
proto 6, length: 40) 10.10.224.6.8001 > 10.1.3.131.1045: F [tcp sum
ok] 4053960032:4053960032(0) ack 1538769655 win 6432
03:15:36.686007 IP (tos 0x0, ttl  64, id 3070, offset 0, flags [DF],
proto 6, length: 40) 10.10.224.6.8001 > 10.1.1.8.4824: F [tcp sum ok]
4120796002:4120796002(0) ack 2440799003 win 10992
03:15:37.229926 IP (tos 0x0, ttl  64, id 60997, offset 0, flags [DF],
proto 6, length: 52) 10.10.224.6.8001 > 10.1.1.86.2288: F [tcp sum ok]
4164262937:4164262937(0) ack 3891303954 win 7497 <nop,nop,timestamp
24260104 27180085>
03:15:37.802145 IP (tos 0x0, ttl  64, id 60433, offset 0, flags [DF],
proto 6, length: 40) 10.10.224.6.8001 > 10.1.1.8.4816: F [tcp sum ok]
4105151875:4105151875(0) ack 189658004 win 6432