Trouble getting server side transparency to work
I am using the following versions of software: - Ubuntu 8.04 (Linux 2.6.24-14) Base Kernel - tproxy-2.6.24-20080509 patch (tproxy-kernel-2.6.24-20080509-164556-1210344356.tar.bz2) - iptables 1.4.0 (from netfilter.org website) - tproxy-iptables-svn-r7519-20080509-165158-1210344718.patch I am using the following script to setup the tproxy rules. ==================================== ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 iptables -t mangle -N DIVERT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 50080 ==================================== The host I am using is running as a router with eth0 connected to internet and eth1 connected to LAN. I have small a tcp proxy program running on this host. I enabled IP_TRANSPARENT on both sides (client side and server side). The client side is working wonderfully fine. It accepts connections from the client (otherwise destined to the real server), can read and write packets through the accepted FD. However, I am having trouble with the server side socket. I successfully bound the socket to a foriegn address (client's ip and port 0) and initiated a connection to the real server. The TCP SYN is going out of the box and SYN ACK is coming in. But this SYN ACK does not make it into the socket layer (or atleast my program) and socket stays in SYN_SENT state until time out. Can you please help me with this? It feels so close but ... Thanks Ravi My iptables output looks like this: ====================== Chain PREROUTING (policy ACCEPT) target prot opt source destination DIVERT tcp -- anywhere anywhere socket TPROXY tcp -- anywhere anywhere tcp dpt:www TPROXY redirect 0.0.0.0:50080 mark 0x1/0x1 Chain INPUT (policy ACCEPT) target prot opt source destination Chain FORWARD (policy ACCEPT) target prot opt source destination Chain OUTPUT (policy ACCEPT) target prot opt source destination Chain POSTROUTING (policy ACCEPT) target prot opt source destination Chain DIVERT (1 references) target prot opt source destination MARK all -- anywhere anywhere MARK set 0x1 ACCEPT all -- anywhere anywhere ==================================== My modules listing looks like this: ==================================== xt_tcpudp 4352 1 xt_MARK 3328 1 xt_multiport 4480 0 xt_socket 4224 1 nf_conntrack 67136 1 xt_socket xt_TPROXY 3712 1 nf_defrag_ipv4 3456 2 xt_socket,xt_TPROXY nf_tproxy_core 5376 2 xt_socket,xt_TPROXY,[permanent] x_tables 16388 6 xt_tcpudp,xt_MARK,xt_multiport,xt_socket,xt_TPROXY,ip_tables ====================================
This is a shot in the dark, but does your server have the correct ARP entry for the client? Unless your proxy is physically inline, the server will have the client machine's MAC for the client IP, and not the proxy MAC. Therefore packets will be sent directly to the client, which is not what you want. In the past I have added the client MAC as a static ARP entry on the server to work around this issue (for testing). Cam Ravi Valmikam wrote:
I am using the following versions of software:
- Ubuntu 8.04 (Linux 2.6.24-14) Base Kernel - tproxy-2.6.24-20080509 patch (tproxy-kernel-2.6.24-20080509-164556-1210344356.tar.bz2) - iptables 1.4.0 (from netfilter.org website) - tproxy-iptables-svn-r7519-20080509-165158-1210344718.patch
I am using the following script to setup the tproxy rules. ==================================== ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 iptables -t mangle -N DIVERT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 50080 ====================================
The host I am using is running as a router with eth0 connected to internet and eth1 connected to LAN.
I have small a tcp proxy program running on this host. I enabled IP_TRANSPARENT on both sides (client side and server side). The client side is working wonderfully fine. It accepts connections from the client (otherwise destined to the real server), can read and write packets through the accepted FD.
However, I am having trouble with the server side socket. I successfully bound the socket to a foriegn address (client's ip and port 0) and initiated a connection to the real server. The TCP SYN is going out of the box and SYN ACK is coming in. But this SYN ACK does not make it into the socket layer (or atleast my program) and socket stays in SYN_SENT state until time out.
Can you please help me with this? It feels so close but ...
Thanks Ravi
My iptables output looks like this: ====================== Chain PREROUTING (policy ACCEPT) target prot opt source destination DIVERT tcp -- anywhere anywhere socket TPROXY tcp -- anywhere anywhere tcp dpt:www TPROXY redirect 0.0.0.0:50080 mark 0x1/0x1
Chain INPUT (policy ACCEPT) target prot opt source destination
Chain FORWARD (policy ACCEPT) target prot opt source destination
Chain OUTPUT (policy ACCEPT) target prot opt source destination
Chain POSTROUTING (policy ACCEPT) target prot opt source destination
Chain DIVERT (1 references) target prot opt source destination MARK all -- anywhere anywhere MARK set 0x1 ACCEPT all -- anywhere anywhere ====================================
My modules listing looks like this: ==================================== xt_tcpudp 4352 1 xt_MARK 3328 1 xt_multiport 4480 0 xt_socket 4224 1 nf_conntrack 67136 1 xt_socket xt_TPROXY 3712 1 nf_defrag_ipv4 3456 2 xt_socket,xt_TPROXY nf_tproxy_core 5376 2 xt_socket,xt_TPROXY,[permanent] x_tables 16388 6 xt_tcpudp,xt_MARK,xt_multiport,xt_socket,xt_TPROXY,ip_tables ==================================== _______________________________________________ tproxy mailing list tproxy@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/tproxy
Ravi Valmikam wrote:
However, I am having trouble with the server side socket. I successfully bound the socket to a foriegn address (client's ip and port 0) and initiated a connection to the real server. The TCP SYN is going out of the box and SYN ACK is coming in. But this SYN ACK does not make it into the socket layer (or atleast my program) and socket stays in SYN_SENT state until time out.
Can you please help me with this? It feels so close but ...
Cameron Schaus wrote: This is a shot in the dark, but does your server have the correct ARP entry for the client? Unless your proxy is physically inline, the server will have the client machine's MAC for the client IP, and not the proxy MAC. Therefore packets will be sent directly to the client, which is not what you want.
In the past I have added the client MAC as a static ARP entry on the server to work around this issue (for testing).
Cam
As far as I know that's an expected behaviour - because the return path is destined for the foreign IP and not the proxy. Adding a static ARP entry works for testing only, as this will result in too many arp entries in a live network. Either setup the system as a bridge ( and hence capable of hijacking the return path ) or if the return path goes through some routers where the routers will route the packets to your proxy. Cheers.
Cameron Thanks for your idea. It was right on the mark. The packet was going to the wrong mac address (directly to the client machine). Once I took care of that issue, everything worked perfectly. This issue was mostly due to Windows's wierd behaviour. Windows my host PC where I was running the two Linux VMs. For some reason windows insists ARPing even a remote IP address that is tunneled/routed through the server VM. If someone else has an issue like this, I would be more than happy to share my experience. Thanks again for your pointer. It definitely helped me out. Thanks Ravi On Mon, May 19, 2008 at 1:34 AM, Cameron Schaus <cam@schaus.ca> wrote:
This is a shot in the dark, but does your server have the correct ARP entry for the client? Unless your proxy is physically inline, the server will have the client machine's MAC for the client IP, and not the proxy MAC. Therefore packets will be sent directly to the client, which is not what you want.
In the past I have added the client MAC as a static ARP entry on the server to work around this issue (for testing).
Cam
Ravi Valmikam wrote:
I am using the following versions of software:
- Ubuntu 8.04 (Linux 2.6.24-14) Base Kernel - tproxy-2.6.24-20080509 patch (tproxy-kernel-2.6.24-20080509-164556-1210344356.tar.bz2) - iptables 1.4.0 (from netfilter.org website) - tproxy-iptables-svn-r7519-20080509-165158-1210344718.patch
I am using the following script to setup the tproxy rules. ==================================== ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 iptables -t mangle -N DIVERT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 50080 ====================================
The host I am using is running as a router with eth0 connected to internet and eth1 connected to LAN.
I have small a tcp proxy program running on this host. I enabled IP_TRANSPARENT on both sides (client side and server side). The client side is working wonderfully fine. It accepts connections from the client (otherwise destined to the real server), can read and write packets through the accepted FD.
However, I am having trouble with the server side socket. I successfully bound the socket to a foriegn address (client's ip and port 0) and initiated a connection to the real server. The TCP SYN is going out of the box and SYN ACK is coming in. But this SYN ACK does not make it into the socket layer (or atleast my program) and socket stays in SYN_SENT state until time out.
Can you please help me with this? It feels so close but ...
Thanks Ravi
My iptables output looks like this: ====================== Chain PREROUTING (policy ACCEPT) target prot opt source destination DIVERT tcp -- anywhere anywhere socket TPROXY tcp -- anywhere anywhere tcp dpt:www TPROXY redirect 0.0.0.0:50080 mark 0x1/0x1
Chain INPUT (policy ACCEPT) target prot opt source destination
Chain FORWARD (policy ACCEPT) target prot opt source destination
Chain OUTPUT (policy ACCEPT) target prot opt source destination
Chain POSTROUTING (policy ACCEPT) target prot opt source destination
Chain DIVERT (1 references) target prot opt source destination MARK all -- anywhere anywhere MARK set 0x1 ACCEPT all -- anywhere anywhere ====================================
My modules listing looks like this: ==================================== xt_tcpudp 4352 1 xt_MARK 3328 1 xt_multiport 4480 0 xt_socket 4224 1 nf_conntrack 67136 1 xt_socket xt_TPROXY 3712 1 nf_defrag_ipv4 3456 2 xt_socket,xt_TPROXY nf_tproxy_core 5376 2 xt_socket,xt_TPROXY,[permanent] x_tables 16388 6 xt_tcpudp,xt_MARK,xt_multiport,xt_socket,xt_TPROXY,ip_tables ==================================== _______________________________________________ tproxy mailing list tproxy@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/tproxy
participants (3)
-
Cameron Schaus
-
Ming-Ching Tiew
-
Ravi Valmikam