- tproxy - lists.balabit.hu

Correct kernel version with tproxy
by Alexandre Correa 11 Jan '10

11 Jan '10

What´s the best version of kernel for using tproxy ? 2.6.{28|29|30|31} ? seems with 2.6.32 has issues with.. true ? regadrs. -- Sds. Alexandre J. Correa Onda Internet http://www.onda.net.br IPV6 Ready !!! http://ipv6.onda.net.br

2 1

Fwd: Funny Tproxy ISSUE with ip route lost communication
by Luiz Biazus 09 Jan '10

09 Jan '10

Thanks for Reply Krisztian after i insert this rules: ip rule add dev eth0 fwmark 1 lookup 100 ip rule add dev eth1 fwmark 1 lookup 100 ip rule add dev br0 fwmark 1 lookup 100 i must to insert this route: ip route add local 0.0.0.0/0 dev lo table 100 or dont? 2010/1/8 KOVACS Krisztian <hidden(a)balabit.hu>: > Hi, > > On 01/07/2010 11:56 AM, Luiz Biazus wrote: >> >> let me explain what is funny >> i have this plataform: >> >> ubuntu 9.04 >> kernel 2.6.32 >> iptables 1.4.6 >> squid 3.1 running in port 8012 > >> >> [...] >> >> until here all is OK even i can UP all this rulles that i discribe >> before and every thing seems to be OK! >> >> BUT! >> >> >> if i up THIS RULES: >> >> ip rule add fwmark 1 lookup 100 >> ip route add local 0.0.0.0/0 dev lo table 100 >> >> I LOST COMMUNICATION WITH MY PROXY > > I'm afraid this was caused by Jamal's rp_filter change committed in Linux > 2.6.32. > > You can find more information on the issue in the linux-netdev or tproxy > mailing list archives: > > http://marc.info/?l=linux-netdev&m=125925598910379&w=2 > > This is expected to be fixed in an upcoming -stable release (2.6.32.z), in > the meantime you can use the workaround mentioned in that mail thread: > instead of the 'ip rule add fwmark 1 lookup 100' rule, you have to use one > rule per interface. > > <quote> > The workaround is using more specific ip rules that include the ingress > interface name: > > # ip rule add dev eth0 fwmark 1 lookup 100 > > (repeat the above for each interface except lo.) > </quote> > > Hope this helps. > > Cheers, > Krisztian >

2 1

ipfw and dummynet on linux
by Luiz Biazus 08 Jan '10

08 Jan '10

Hello there http://info.iet.unipi.it/~luigi/dummynet/ it may help to implement what do you think?! Bst Rgds Luiz Biazus

1 0

Funny Tproxy ISSUE with ip route lost communication
by Luiz Biazus 08 Jan '10

08 Jan '10

Hi There let me explain what is funny i have this plataform: ubuntu 9.04 kernel 2.6.32 iptables 1.4.6 squid 3.1 running in port 8012 iface ETH0 with no ip configured (connected in a gw ip 189.10.205.1) iface ETH1 with no ip configured(all the clients behind this interface 189.10.205.0/24) br0 with ip adress 189.10.205.3 with interface ETH0 and ETH1 associed echo 0 > /proc/sys/net/ipv4/conf/lo/rp_filter echo 1 > /proc/sys/net/ipv4/ip_forward echo 1 > /proc/sys/net/ipv4/ip_nonlocal_bind echo 0 > /proc/sys/net/ipv4/conf/eth1/rp_filter echo 0 > /proc/sys/net/ipv4/conf/eth0/rp_filter echo 0 > /proc/sys/net/ipv4/conf/br0/rp_filter echo 1 > /proc/sys/net/ipv4/conf/all/forwarding echo 1 > /proc/sys/net/ipv4/conf/all/send_redirects echo 1 > /proc/sys/net/ipv4/conf/eth0/send_redirects iptables -t mangle -N DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 8012 ebtables -t broute -A BROUTING -i eth1 -p ipv4 --ip-proto tcp --ip-dport 80 -j redirect --redirect-target ACCEPT (if i use DROP stop to navegate) ebtables -t broute -A BROUTING -i eth0 -p ipv4 --ip-proto tcp --ip-sport 80 -j redirect --redirect-target ACCEPT (if i use DROP stop to navegate) brctl stp br0 off brctl setfd br0 1 brctl sethello br0 1 cd /proc/sys/net/bridge/ for i in * do echo 0 > $i done until here all is OK even i can UP all this rulles that i discribe before and every thing seems to be OK! BUT! if i up THIS RULES: ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 I LOST COMMUNICATION WITH MY PROXY all my clients stop to browsing (even me cuz im behind)... i cannot use this in my netwok layout? iven i cant ssh to the proxy, just ping i think that routes getting crazy... other stuff: lo Link encap:Loopback Local inet end.: 127.0.0.1 Masc:255.0.0.0 endereÃ§o inet6: ::1/128 Escopo:MÃ¡quina UP LOOPBACK RUNNING MTU:16436 MÃ©trica:1 pacotes RX:0 erros:0 descartados:0 excesso:0 quadro:0 Pacotes TX:0 erros:0 descartados:0 excesso:0 portadora:0 colisÃµes:0 txqueuelen:0 RX bytes:0 (0.0 B) TX bytes:0 (0.0 B) my LO is allways with ZERO BYTES... someone!? Thanks Best Rgds Luiz Biazus

2 1

squid between ciscos using wccp2
by hugo＠gigared.com 03 Dec '09

03 Dec '09

hi, im working on a wccp2+tproxy4+squid transparent cache platform. my issue is trying to put two or more squids receiving the clients packet from clients via a cisco 3560 using wccp2 but the output if squids boxes to the internet are via other 3560. the question is: its reasonable configure wccp2 on the output of squids to the second cisco? a mininal diagram: -> squid1 <- clients <-> cisco(wccpv2) <--> squidn <-->cisco(wccp2)<--> internet regards Hugo Petrucci ------------------------- www.gigared.com

1 0

Re: [tproxy] [tproxy,regression] tproxy broken in 2.6.32
by KOVACS Krisztian 03 Dec '09

03 Dec '09

Hi, On Thu, 2009-12-03 at 14:55 +0100, Patrick McHardy wrote: > jamal wrote: > > BTW, it should be noted that the change from Patrick to fib_validate > > which allows to accept local routes from will also solve this problem. > > My suggestion below is to restore old expected behavior.. > > Agreed, the accept_local sysctl should not be misused for this, > otherwise TPROXY setups wouldn't have source validation anymore. Absolutely agreed. Cheers, Krisztian

1 0

[HELP] Tproxy server Can't receive any client packet
by 박제호 02 Dec '09

02 Dec '09

Hello everyone i have a problem in my transparent proxy test, i recently made up the testbed as below to run the tproxy patched apache proxy [mod_proxy], and i applied all iptables and routing rules with referencing the readme file [http://www.balabit.com/downloads/files/tproxy/README.txt, http://www.mjmwired.net/kernel/Documentation/networking/tproxy.txt] the proxy server listening the port 3128 and i checked there were no problem. but when the client tried to connect the web server, the packets reached to the box and i found the usage counts of filter rules in the mangle table incresed but my tproxy server could not receive any corresponding packet from the socket I want to know why my proxy server can't receive any packet through the socket, Do i need some more DNAT rules ? ---------------- testbed --------------- [client ] <-----> [tproxy patched apache mod_proxy] <-----> [web server] --------------------------------------------------- proxybox [Environment of the tproxy server] 1. kernel kernel 2.6.24 + tproxy-kernel-2.6.24-20080602-165651-1212418611.tar.bz2 [http://www.balabit.com/downloads/files/tproxy/tproxy-kernel-2.6.24-20080602…] 2. iptables iptables 4.0 + tproxy-iptables-1.4.0-20080521-113954-1211362794.patc [http://www.balabit.com/downloads/files/tproxy/tproxy-iptables-1.4.0-2008052…] 3. apache apache 2.2.9 + httpd-2.2.9-tproxy.patch [i got this patch from google and ported it to this version ] here are the config files of the kernel and apache proxy

3 3

Re: [tproxy] SO_BINDTODEVICE and IP_TRANSPARENT (TPROXY)
by Ashwani Wason 02 Dec '09

02 Dec '09

After debugging a bit into how nf_tproxy_get_sock_v4() looks up a listener for foreign connections using __inet_lookup_listener() and compute_score() I found that the problem was happening because of the way I had the TPROXY rules setup. Those rules work if the proxy has a single listening socket for INADDR_ANY. If multiple listening sockets must be used, one for each local address, which is the case for using SO_BINDTODEVICE then the TPROXY rules must also be "fully qualified" (with interface name [-i] and IP address thereof [--on-ip]). So the rules in my example change as follows... Instead of using: > iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 --on-port 4002 Use: iptables -t mangle -A PREROUTING *-i eth0* -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 *--on-ip 192.168.0.65* --on-port 4002 iptables -t mangle -A PREROUTING *-i eth1* -p tcp --dport 80 -j TPROXY --tproxy-mark 0x1/0x1 *--on-ip 192.168.1.65* --on-port 4002 With these rules the listening socket was looked up correctly and the sk_bound_dev_if of the socket was honored. - Ashwani PS: Sorry to anyone (Jamal) who already spent any time on this. PPS: Copying the tproxy list in case someone else is looking for this stuff in the future. On Mon, Nov 30, 2009 at 3:15 PM, Ashwani Wason <ashwas(a)gmail.com> wrote: > It appears that SO_BINDTODEVICE is not working when used along with > IP_TRANSPARENT of TPROXY. The use case is for a transparent proxy that > has to handle clients with overlapping IP addresses coming in on > different interfaces (VLANs or physical). Each such interface has a > unique next hop gateway. > > Setup: Kernel 2.6.30.9 built with required TPROXY configuration. A > system with three interfaces, eth0 (192.168.0.65/24), eth0.1 > (10.0.7.65/16), and eth1 (192.168.1.65/24), each with its own default > router and a fairly standard set of TPROXY rules: > > default via 192.168.1.62 dev eth1 > default via 192.168.0.62 dev eth0 > default via 10.0.0.9 dev eth0.1 > > iptables -t mangle -N DIVERT > iptables -t mangle -A DIVERT -j MARK --set-mark 1 > iptables -t mangle -A DIVERT -j ACCEPT > iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT > iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY > --tproxy-mark 0x1/0x1 --on-port 4002 > > ip rule add fwmark 1 iif eth0 lookup 101 > ip rule add fwmark 1 iif eth1 lookup 101 > > ip route add local 0.0.0.0/0 dev lo table 101 > > Additional information: > > * The proxy has a separate listening socket for each interface. > > * On each listening socket the proxy sets the IP_TRANSPARENT option. > It also sets the SO_BINDTODEVICE option (on each socket) to the > associated interface. (I have experimented with the ordering of these > options and setting one before the other has no change in behavior.) > > * The proxy works perfectly fine in "transparent mode" when > SO_BINDTODEVICE is not used. > > * The proxy works perfectly fine in "explicit mode" when > SO_BINDTODEVICE *is used*. > > Now my test is very simple with the expectation that when a SYN comes > in on, say, interface eth0 then the SYN/ACK must go out back on that > interface to the MAC address of the designated next hop router. That > does not seem to happen however. The SYN/ACK is not being sent out at > all, from any interface. It seems that either the SYN or the SYN/ACK > is being eaten up somewhere. No one 'netstat -s' seems to be updated > either. (Since even 'netstat -ant' does not show the TCP connection > state, I suspect that the SYN is being eaten up.) > > When I reconfigure the proxy to be in explicit mode, i.e. not set the > IP_TRANSPARENT option, and have the clients connect directly to the IP > address and port of the proxy then the system works as expected, i.e. > the SYN/ACK (and all subsequent packets) go out on the associated > interface. > > Thank you, > Ashwani >

1 0

Re: [tproxy] [tproxy,regression] tproxy broken in 2.6.32
by KOVACS Krisztian 30 Nov '09

30 Nov '09

Hi, On Mon, 2009-11-30 at 07:15 -0500, jamal wrote: > On Sun, 2009-11-29 at 21:35 +0100, KOVACS Krisztian wrote: > > > The story is that we really do want to deliver these packets locally, as > > if the destination IP address was locally configured on the host. The only > > way I know of to get the packet to ip_local_deliver() is by using a local > > route. > > Aha, now i understand where both you and Patrick are coming from. So > you literally have to hit the main(or default) table in the reverse > source validation. How does the workaround (that you suggested) work > then? i.e you are going to fail the RTN_UNICAST test no matter what. No, because by narrowing the rule to specific ingress interfaces the lookup done in fib_validate_source() won't match the rule(s) (because the flow used will have iif set to the loopback device), and thus it will look up the main table and select a unicast route. > Dave, give me some short time to mull this over. I am not sure i like > the sysctl approach - we may have to just revert the whole thing > instead. I don't think it would be unreasonable to add a sysctl but disable the feature by default. It's up to you, of course. Cheers, Krisztian

1 0

Re: [tproxy] [tproxy,regression] tproxy broken in 2.6.32
by KOVACS Krisztian 29 Nov '09

29 Nov '09

Hi, On szo, nov 28, 2009 at 02:44:02 -0500, jamal wrote: > > The source address *is* unicast. > > Sorry - I meant the route type is unicast. The fact that an address is > unicast or not is already dealt with by the time you get to source > address validation (in ip_input()) > > > The problem is that the routing setup is > > asymmetrical, as Patrick has already mentioned: we're using a mark to > > force certain packets (those that have matching sockets on the host) being > > delivered locally. > > > > In the other direction, reply packets won't be marked by the iptables > > rules and thus will be routed on egress just fine. > > In that case i dont understand the reluctance to use unicast routes. > Maybe you can explain and put me at ease because i see youve put extra > effort to use local addresses. The short answer is: it doesn't work with unicast routes. The story is that we really do want to deliver these packets locally, as if the destination IP address was locally configured on the host. The only way I know of to get the packet to ip_local_deliver() is by using a local route. (Now that you mentioned this I've actually gave it a try. Changing 'local' in the route to 'unicast' doesn't work at all: incoming packets get dropped because forwarding is not enabled on the ingress interface.) -- KOVACS Krisztian

1 0