[tproxy] squid, cttproxy, and a redirector script
Wayne Smith
packetbl@allofmy.info
Tue, 5 Apr 2005 09:18:25 -0400
I'm not sure what the best list for this issue is, so I'm starting here. If I got it wrong, please point me in the right direction after your done flaming ;)
Background:
----------------
Fedora FC2 box, kernel 2.6.10 with AC and tproxy patches applied
Squid 2.5 Stable 5, transparent proxy patch applied.
ebtables and iptables
All the above is working just fine and transparent redirection is happening without issue. The origin webserver sees the IP address of the client and life is pretty happy.
Upon noticing that Microsoft is making it, shall we say hard?, to cache windowsupdates, I stumbled across http://www.glob.com.au/windowsupdate_cache/
It's a set of three perl scripts. It basically will wget a windows patch, store it on a local server, and then in the future rewrite the requests from squid so that the content is pulled locally. redir.pl does the redirection part.
redir.pl is working and kicks out the right URL
/usr/local/bin/redir.pl
download.microsoft.com/download/1/2/a/12a31f29-2fa9-4f50-b95d-e45ef7013f87/MP10Setup.exe 192.168.1.1 bob GET
get's rewrote to
http://127.0.0.1/cache/store/download.microsoft.com/download/1/2/a/12a31f29-2fa9-4f50-b95d-e45ef7013f87/MP10Setup.exe
The problem is the file never comes down. I've also tried redir.pl rewriting the URI to use the boxes legitimate IP. Same thing. Eventually you get the squid timeout.
the ebtables and iptables rules in use
---------------------------------------
ebtables -t broute -A BROUTING -i eth1 -p IPv4 --ip-protocol 6 --ip-destination-port 80 --ip-source $1 --ip-destination ! x.x.x.0/24 -j redirect --redirect-target ACCEPT
iptables -t nat -A PREROUTING -i br0 -p tcp --dport 80 -s $1 -d ! x.x.x.0/24 -j REDIRECT --to-port 3128
Pretty straight forward... if it's IP and destined for TCP port 80, let IPTABLES see it (unless it's aimed for a local network). Iptables takes it and ships it to port 3128 for squid to party. This is working great for 'just' going through the proxy and hitting remote webservers.
While working on getting the redirector script and everything to work, I was for some reason able to push a few files out (not sure how in testing they got wedged into working).
Here's where 2003 sp1 gets redirected and pulled
redir.pl's log
---------------
[Tue Apr 5 09:00:32 2005] cachehit 345322744 download.microsoft.com/download/1/2/7/127c5938-d36a-4405-9df1-f00d57495652/WindowsServer2003-KB889101-SP1-x86-ENU.exe
tethereal -i lo
--------------------
Capturing on lo
0.000000 127.0.0.1 -> 127.0.0.1 TCP 46624 > 46623 [PSH, ACK] Seq=0 Ack=0 Win=8192 Len=147 TSV=2861348298 TSER=2858781735
0.016805 127.0.0.1 -> 127.0.0.1 TCP 46623 > 46624 [PSH, ACK] Seq=0 Ack=147 Win=8192 Len=147 TSV=2861348315 TSER=2861348298
0.016830 127.0.0.1 -> 127.0.0.1 TCP 46624 > 46623 [ACK] Seq=147 Ack=147 Win=8192 Len=0 TSV=2861348315 TSER=2861348315
and the file comes down at a health 7 MB/second.
Here's the same info with the process breaking. In this case, a copy of Media Player 10
0.000000 127.0.0.1 -> 127.0.0.1 TCP 46624 > 46623 [PSH, ACK] Seq=0 Ack=0 Win=8192 Len=118 TSV=2861551908 TSER=2861348315
0.000357 127.0.0.1 -> 127.0.0.1 TCP 46623 > 46624 [PSH, ACK] Seq=0 Ack=118 Win=8192 Len=118 TSV=2861551909 TSER=2861551908
0.000378 127.0.0.1 -> 127.0.0.1 TCP 46624 > 46623 [ACK] Seq=118 Ack=118 Win=8192 Len=0 TSV=2861551909 TSER=2861551909
0.000656 x.x.x.98 -> 127.0.0.1 TCP 56643 > http [SYN] Seq=0 Ack=0 Win=32767 Len=0 MSS=16396 TSV=2861551909 TSER=0 WS=2
3.000252 x.x.x.98 -> 127.0.0.1 TCP 56643 > http [SYN] Seq=0 Ack=0 Win=131068 Len=0 MSS=16396 TSV=2861554909 TSER=0 WS=2
8.999440 x.x.x.98 -> 127.0.0.1 TCP 56643 > http [SYN] Seq=0 Ack=0 Win=131068 Len=0 MSS=16396 TSV=2861560909 TSER=0 WS=2
20.997818 x.x.x.98 -> 127.0.0.1 TCP 56643 > http [SYN] Seq=0 Ack=0 Win=131068 Len=0 MSS=16396 TSV=2861572909 TSER=0 WS=2
44.993570 x.x.x.98 -> 127.0.0.1 TCP 56643 > http [SYN] Seq=0 Ack=0 Win=131068 Len=0 MSS=16396 TSV=2861596909 TSER=0 WS=2
59.107596 x.x.x.98 -> 127.0.0.1 TCP 57640 > http [SYN] Seq=0 Ack=0 Win=32767 Len=0 MSS=16396 TSV=2861611025 TSER=0 WS=2
62.107252 x.x.x.98 -> 127.0.0.1 TCP 57640 > http [SYN] Seq=0 Ack=0 Win=131068 Len=0 MSS=16396 TSV=2861614025 TSER=0 WS=2
68.106443 x.x.x.98 -> 127.0.0.1 TCP 57640 > http [SYN] Seq=0 Ack=0 Win=131068 Len=0 MSS=16396 TSV=2861620025 TSER=0 WS=2
80.104816 x.x.x.98 -> 127.0.0.1 TCP 57640 > http [SYN] Seq=0 Ack=0 Win=131068 Len=0 MSS=16396 TSV=2861632025 TSER=0 WS=2
104.100569 x.x.x.98 -> 127.0.0.1 TCP 57640 > http [SYN] Seq=0 Ack=0 Win=131068 Len=0 MSS=16396 TSV=2861656025 TSER=0 WS=2
119.118652 x.x.x.98 -> 127.0.0.1 TCP 58643 > http [SYN] Seq=0 Ack=0 Win=32767 Len=0 MSS=16396 TSV=2861671045 TSER=0 WS=2
122.118137 x.x.x.98 -> 127.0.0.1 TCP 58643 > http [SYN] Seq=0 Ack=0 Win=131068 Len=0 MSS=16396 TSV=2861674045 TSER=0 WS=2
128.117322 x.x.x.98 -> 127.0.0.1 TCP 58643 > http [SYN] Seq=0 Ack=0 Win=131068 Len=0 MSS=16396 TSV=2861680045 TSER=0 WS=2
140.115697 x.x.x.98 -> 127.0.0.1 TCP 58643 > http [SYN] Seq=0 Ack=0 Win=131068 Len=0 MSS=16396 TSV=2861692045 TSER=0 WS=2
164.111456 x.x.x.98 -> 127.0.0.1 TCP 58643 > http [SYN] Seq=0 Ack=0 Win=131068 Len=0 MSS=16396 TSV=2861716045 TSER=0 WS=2
redir.pl's log entry
[Tue Apr 5 09:03:55 2005] cachehit 12754672 download.microsoft.com/download/1/2/a/12a31f29-2fa9-4f50-b95d-e45ef7013f87/MP10Setup.exe
Squid's error
"ERROR
The requested URL could not be retrieved
--------------------------------------------------------------------------------
While trying to retrieve the URL: http://127.0.0.1/cache/store/download.microsoft.com/download/1/2/a/12a31f29-2fa9-4f50-b95d-e45ef7013f87/MP10Setup.exe
The following error was encountered:
Connection Failed
The system returned:
(110) Connection timed out"
here's that same failed request with the redir.pl giving the external IP
tethereal -i lo
Capturing on lo
0.000000 127.0.0.1 -> 127.0.0.1 TCP 37898 > 37897 [PSH, ACK] Seq=0 Ack=0 Win=8192 Len=118 TSV=2862173433 TSER=2862151017
0.001160 127.0.0.1 -> 127.0.0.1 TCP 37897 > 37898 [ACK] Seq=0 Ack=118 Win=8192 Len=0 TSV=2862173434 TSER=2862173433
0.001559 127.0.0.1 -> 127.0.0.1 TCP 37897 > 37898 [PSH, ACK] Seq=0 Ack=118 Win=8192 Len=121 TSV=2862173434 TSER=2862173433
0.001569 127.0.0.1 -> 127.0.0.1 TCP 37898 > 37897 [ACK] Seq=118 Ack=121 Win=8192 Len=0 TSV=2862173434 TSER=2862173434
0.001950 x.x.x.98 -> x.x.y.7 TCP 38323 > http [SYN] Seq=0 Ack=0 Win=32767 Len=0 MSS=16396 TSV=2862173435 TSER=0 WS=2
3.002299 x.x.x.98 -> x.x.y.7 TCP 38323 > http [SYN] Seq=0 Ack=0 Win=131068 Len=0 MSS=16396 TSV=2862176435 TSER=0 WS=2
9.001486 x.x.x.98 -> x.x.y.7 TCP 38323 > http [SYN] Seq=0 Ack=0 Win=131068 Len=0 MSS=16396 TSV=2862182435 TSER=0 WS=2
20.998860 x.x.x.98 -> x.x.y.7 TCP 38323 > http [SYN] Seq=0 Ack=0 Win=131068 Len=0 MSS=16396 TSV=2862194435 TSER=0 WS=2
It's like the socket isn't happening for some reason. The connection never happens and the process times out with the same squid error (only now with the external IP in the
unable to retrieve URL). The apache webserver logs don't show anything for the sessions that timeout. The client can pull up the apache webpage when directly typed in.
Any thoughts on how to get this process to cooperate or how to continue troubleshooting would be appreciated! I've been staring at it a bit and I'm stumped.
Wayne