On Tue, 2010-03-02 at 16:23 +0100, nicolas normand wrote:
Hi
It is possible for a proxy to transparently and simultaneously connect to several backends (with different IP) with the same client IP/Port ? I have tried to SO_REUSEADDR before binding but I have a strange behavior:
* the concerned program is a tproxy apache patch, which works perfectly for simple and single request, but doesn't handle subrequests (and probably some other twisted cases). * the patch is based on this incomplete one: http://miscfiles.googlecode.com/svn/trunk/tproxy.patch , which I just ported it to apache 2.2.14, made the listening socket transparent and made the options to work. You can found it included this mail. * Code in proxy/module/proxy_utils.c now look likes: if (conf->tproxy) { if ((rv = apr_socket_opt_set(newsock, APR_IP_TRANSPARENT, 1)) != APR_SUCCESS) { ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, "apr_socket_opt_set(APR_IP_TRANSPARENT): Failed to set"); }
if ((rv = apr_socket_opt_set(newsock, APR_SO_REUSEADDR, 1)) != APR_SUCCESS) { ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, "apr_socket_opt_set(APR_SO_REUSE_ADDR): Failed to set"); }
if ((rv = apr_socket_bind(newsock, remote_addr)) != APR_SUCCESS) { ap_log_error(APLOG_MARK, APLOG_ERR, rv, s, "proxy: error binding to socket %pI", remote_addr); return HTTP_INTERNAL_SERVER_ERROR; } }
When connecting the second backend (which has a different IP address), I got this (192.168.100.12 is the CLIENT, 192.168.100.127 is the PROXY, 192.168.100.238 and 192.168.100.229 are the BACKENDS):
// Connecting to first backend, all is okay 15:49:32.037788 IP 192.168.100.12.57009 > 192.168.100.238.80: S 1617296302:1617296302(0) win 5840 <mss 1460,sackOK,timestamp 21314512 0,nop,wscale 6> 15:49:32.037827 IP 192.168.100.238.80 > 192.168.100.12.57009: S 1072906513:1072906513(0) ack 1617296303 win 5792 <mss 1460,sackOK,timestamp 107453675 21314512,nop,wscale 4> 15:49:32.038268 IP 192.168.100.12.57009 > 192.168.100.238.80: . ack 1 win 92 <nop,nop,timestamp 21314512 107453675> 15:49:32.038855 IP 192.168.100.12.57009 > 192.168.100.238.80: P 1:386(385) ack 1 win 92 <nop,nop,timestamp 21314512 107453675> 15:49:32.038877 IP 192.168.100.238.80 > 192.168.100.12.57009: . ack 386 win 429 <nop,nop,timestamp 107453676 21314512> 15:49:32.040221 IP 192.168.100.238.80 > 192.168.100.12.57009: . 1:1449(1448) ack 386 win 429 <nop,nop,timestamp 107453676 21314512> 15:49:32.040377 IP 192.168.100.238.80 > 192.168.100.12.57009: P 1449:1795(346) ack 386 win 429 <nop,nop,timestamp 107453676 21314512> 15:49:32.040485 IP 192.168.100.12.57009 > 192.168.100.238.80: . ack 1449 win 137 <nop,nop,timestamp 21314512 107453676> 15:49:32.040608 IP 192.168.100.12.57009 > 192.168.100.238.80: . ack 1795 win 182 <nop,nop,timestamp 21314513 107453676>
// And now try connecting the second backend... Hell, the client sent the Reset flag... 15:49:32.067711 IP 192.168.100.12.57009 > 192.168.100.229.80: S 1618236269:1618236269(0) win 5840 <mss 1460,sackOK,timestamp 21314519 0,nop,wscale 6> 15:49:32.067861 IP 192.168.100.229.80 > 192.168.100.12.57009: S 840737942:840737942(0) ack 1618236270 win 5792 <mss 1460,sackOK,timestamp 107448908 21314519,nop,wscale 4> 15:49:32.068837 IP 192.168.100.12.57009 > 192.168.100.229.80: R 1618236270:1618236270(0) win 0 15:49:35.069466 IP 192.168.100.12.57009 > 192.168.100.229.80: S 1618236269:1618236269(0) win 5840 <mss 1460,sackOK,timestamp 21315270 0,nop,wscale 6> 15:49:35.069483 IP 192.168.100.229.80 > 192.168.100.12.57009: S 887635567:887635567(0) ack 1618236270 win 5792 <mss 1460,sackOK,timestamp 107449658 21315270,nop,wscale 4> 15:49:35.072257 IP 192.168.100.12.57009 > 192.168.100.229.80: R 1618236270:1618236270(0) win 0 15:49:41.070224 IP 192.168.100.12.57009 > 192.168.100.229.80: S 1618236269:1618236269(0) win 5840 <mss 1460,sackOK,timestamp 21316770 0,nop,wscale 6> 15:49:41.070363 IP 192.168.100.229.80 > 192.168.100.12.57009: S 981396815:981396815(0) ack 1618236270 win 5792 <mss 1460,sackOK,timestamp 107451158 21316770,nop,wscale 4> 15:49:41.071604 IP 192.168.100.12.57009 > 192.168.100.229.80: R 1618236270:1618236270(0) win 0
I have also tried to make a client to simultaneously connect to two different backends while both connecting sockets binded to the same ip (no proxy, no transparency) with no problems... I don't understand what I have missed
Hm.. it should work. I've just tried it with a local virtual machine and it does indeed work. I simulated what you did with two netcat instances, using the same local port and a different destination port (instead of different IPs I've used different ports) $ nc -T -s 10.30.1.33 -p 10000 10.30.1.1 2000 $ nc -T -s 10.30.1.33 -p 10000 10.30.1.1 2001 I've also confirmed that the netcat instances indeed use the same local endpoint, here are the SYN packets: 1) one connection 10:30:20.356370 08:00:27:44:3b:03 > 0a:00:27:00:00:00, ethertype IPv4 (0x0800), length 74: 10.30.1.33.10000 > 10.30.1.1.2001: Flags [S], seq 917488010, win 5840, options [mss 1460,sackOK,TS val 117738 ecr 0,nop,wscale 5], length 0 2) the other connection 10:30:23.589674 08:00:27:44:3b:03 > 0a:00:27:00:00:00, ethertype IPv4 (0x0800), length 74: 10.30.1.33.10000 > 10.30.1.1.2000: Flags [S], seq 960239781, win 5840, options [mss 1460,sackOK,TS val 118542 ecr 0,nop,wscale 5], length 0 So indeed this should work and I guess there's something in your environment that prevents it from working. -- Bazsi