Configuration for multiple listening sockets, none of them on 'lo'
Hi All, I have a transparent proxy, which I want to listen on four VLAN interfaces on 2.6.30 kernel. Various reasons, but the proxy cannot listen on 'lo' or on INADDR_ANY. The interfaces are eth0.31 through eth0.34 and the proxy opens one listening socket per interface. All listening sockets are bound to the same port, 2345. I am trying to set up rules similar to what has been documented so far. This is what I have - the idea is to mark all packets identically (with mark 1) based on destination port 80 but using IP rules have them lookup a different routing table based on the incoming interface. The routing table would then deliver the packet to the corresponding physical interface. The problem is that the proxy never sees these connections (netstat –ant also does not show them, which means that IP is not delivering them).
"iptables -t mangle -N DIVERT" "iptables -t mangle -A DIVERT -j MARK --set-mark 1" "iptables -t mangle -A DIVERT -j ACCEPT" "iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT" "iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 2345"
"ip rule add fwmark 1 iif eth0.31 lookup 101" "ip route add local 0.0.0.0/0 dev eth0.31 table 101"
"ip rule add fwmark 1 iif eth0.32 lookup 102" "ip route add local 0.0.0.0/0 dev eth0.32 table 102"
"ip rule add fwmark 1 iif eth0.33 lookup 103" "ip route add local 0.0.0.0/0 dev eth0.33 table 103"
"ip rule add fwmark 1 iif eth0.34 lookup 104" "ip route add local 0.0.0.0/0 dev eth0.34 table 104"
There is not much in the man page about the 'dev' argument of 'ip route add' other than that it is the output device name. Perhaps my interpretation is incorrect. The IP tables rules are working okay, as the "hit count" of pkts and bytes keeps going up in the output of 'iptables -t mangle -L -nv'. I am not sure what is happening thereafter. The packets are seemingly going into a black hole because 'netstat -sw' does not show them discarded, delivered, or forwarded. [root@]# netstat -sw Ip: 1859156 total packets received 0 forwarded 18 with unknown protocol 0 incoming packets discarded 3917 incoming packets delivered 2692 requests sent out Can you please help identify what is going on? If this has already been posted elsewhere then please let me know. Thanks, - Ashwani
Hi, On sze, júl 29, 2009 at 05:57:32 -0700, Ashwani Wason wrote:
Hi All,
I have a transparent proxy, which I want to listen on four VLAN interfaces on 2.6.30 kernel. Various reasons, but the proxy cannot listen on 'lo' or on INADDR_ANY. The interfaces are eth0.31 through eth0.34 and the proxy opens one listening socket per interface. All listening sockets are bound to the same port, 2345. I am trying to set up rules similar to what has been documented so far. This is what I have - the idea is to mark all packets identically (with mark 1) based on destination port 80 but using IP rules have them lookup a different routing table based on the incoming interface. The routing table would then deliver the packet to the corresponding physical interface. The problem is that the proxy never sees these connections (netstat –ant also does not show them, which means that IP is not delivering them).
"iptables -t mangle -N DIVERT" "iptables -t mangle -A DIVERT -j MARK --set-mark 1" "iptables -t mangle -A DIVERT -j ACCEPT" "iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT" "iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 2345"
"ip rule add fwmark 1 iif eth0.31 lookup 101" "ip route add local 0.0.0.0/0 dev eth0.31 table 101"
"ip rule add fwmark 1 iif eth0.32 lookup 102" "ip route add local 0.0.0.0/0 dev eth0.32 table 102"
"ip rule add fwmark 1 iif eth0.33 lookup 103" "ip route add local 0.0.0.0/0 dev eth0.33 table 103"
"ip rule add fwmark 1 iif eth0.34 lookup 104" "ip route add local 0.0.0.0/0 dev eth0.34 table 104"
Instead of using multiple IP rules and routing tables, I'd suggest using multiple TPROXY targets, plus make use of the --on-ip argument of the TPROXY target. Something like this: iptables -t mangle -A PREROUTING -i eth0.31 -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-ip PROXY_IP_FOR_VLAN_31 --on-port 2345 iptables -t mangle -A PREROUTING -i eth0.32 ... And stick to the recommended routing setup (just one extra routing table and one route). -- KOVACS Krisztian
Thank you - that works. I thought (because of the same mark being used for tproxy and fwmark) that the policy routing rules are required even for transparent capture. - Ashwani On Fri, Jul 31, 2009 at 3:01 AM, KOVACS Krisztian<hidden@sch.bme.hu> wrote:
Hi,
On sze, júl 29, 2009 at 05:57:32 -0700, Ashwani Wason wrote:
Hi All,
I have a transparent proxy, which I want to listen on four VLAN interfaces on 2.6.30 kernel. Various reasons, but the proxy cannot listen on 'lo' or on INADDR_ANY. The interfaces are eth0.31 through eth0.34 and the proxy opens one listening socket per interface. All listening sockets are bound to the same port, 2345. I am trying to set up rules similar to what has been documented so far. This is what I have - the idea is to mark all packets identically (with mark 1) based on destination port 80 but using IP rules have them lookup a different routing table based on the incoming interface. The routing table would then deliver the packet to the corresponding physical interface. The problem is that the proxy never sees these connections (netstat –ant also does not show them, which means that IP is not delivering them).
"iptables -t mangle -N DIVERT" "iptables -t mangle -A DIVERT -j MARK --set-mark 1" "iptables -t mangle -A DIVERT -j ACCEPT" "iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT" "iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 2345"
"ip rule add fwmark 1 iif eth0.31 lookup 101" "ip route add local 0.0.0.0/0 dev eth0.31 table 101"
"ip rule add fwmark 1 iif eth0.32 lookup 102" "ip route add local 0.0.0.0/0 dev eth0.32 table 102"
"ip rule add fwmark 1 iif eth0.33 lookup 103" "ip route add local 0.0.0.0/0 dev eth0.33 table 103"
"ip rule add fwmark 1 iif eth0.34 lookup 104" "ip route add local 0.0.0.0/0 dev eth0.34 table 104"
Instead of using multiple IP rules and routing tables, I'd suggest using multiple TPROXY targets, plus make use of the --on-ip argument of the TPROXY target.
Something like this:
iptables -t mangle -A PREROUTING -i eth0.31 -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-ip PROXY_IP_FOR_VLAN_31 --on-port 2345 iptables -t mangle -A PREROUTING -i eth0.32 ...
And stick to the recommended routing setup (just one extra routing table and one route).
-- KOVACS Krisztian
participants (2)
-
Ashwani Wason
-
KOVACS Krisztian