February 2008 - tproxy - lists.balabit.hu

Re: [tproxy] Squid with tproxy extra brief FAQ - take 2
by Eduardo Schoedler 07 Mar '08

07 Mar '08

Thanks for the FAQ. I'm using the (B) Version Tproxy 4.0.x. However, I haven't found the patch for squid in the site http://www.balabit.hu/downloads/files/tproxy/. I'm using SQUID-2.6.17 with "--enable-linux-tproxy". But this compile options activates suppor for tproxy2 instead tproxy4.0.x, right ? How can I found the patch ? Thanks in advance! Best Regads, Eduardo Schoedler. -------------------------------------------------- From: "Ming-Ching Tiew" <mingching.tiew(a)redtone.com> Subject: [tproxy] Squid with tproxy extra brief FAQ - take 2 1. There are at least 3 different versions of tproxy kernel patches. Each tproxy kernel patch is quite strongly tied to a kernel version, (A) Version Tproxy2 ============= For kernel 2.6.18 URL: http://www.balabit.hu/downloads/files/tproxy/obsolete/ (B) Version Tproxy 4.0.x ================ For kernel 2.6.22 URL: http://www.balabit.hu/downloads/files/tproxy/ (C) Version Tproxy-4.1.0 ================= For kernel 2.6.25 URL: The "official website" is for kernel <=2.6.24 http://people.netfilter.org/hidden/tproxy but the actual version of tproxy 4.1 for 2.6.25 is here: http://people.balabit.hu/panther/tproxy The kernel patch might work with nearby kernel versions, for example, tproxy2 might work with kernel 2.6.19; however it will not work will kernel 2.6.22 ( unless you port it ). 2. Do not confuse tproxy kernel patch mentioned above with squid user-space patches. So far the Squid ( 3.0 and 2.6 ) is only supporting on tproxy2 - the userspace code is integrated. If you managed to compile Squid without changing the source, perhaps with only minor changes in header files, meaning you are likely either did not successfully link in tproxy support or at best it is using tproxy2, and it will not work with tproxy-4.0.x and tproxy-4.1.0 kernel counterpart. However, if you patch the squid source, you should be able to get squid to work with tproxy-4.0.x and tproxy-4.1.0. You can look through the archive of this maillist to look at how to port squid versions to support tproxy-4.0.x and tproxy-4.1.0. Most of the patches floating around are not fully satisfactory, but it could work, at least; but perhaps it will require you to have some programming knowledge. Here maybe a good start :- 3. All the tproxy kernel patches are not compatible with one another. Each requires it's own way of setup and usage. So before doing anything, check if you have gotten the correct info/tproxy version/patches. These are some of the info :- (A) Version Tproxy2 ============ The Squid documentation recommends this :- ebtables -t broute -A BROUTING -p ipv4 --ip-protocol tcp \ --ip-destination-port 80 -j redirect --redirect-target ACCEPT This rule will "broute" bridge traffic from br0 to netfilter. The iptables rule will bring http traffic into local process :- iptables -t tproxy -A PREROUTING -i br0 -p tcp --dport 80 \ -j TPROXY --on-port 3128 To get SNAT working for tproxy2, there is a need for double NAT, and here was the discussion and patch :- https://lists.balabit.hu/pipermail/tproxy/2007-October/000537.html (B) Version tproxy-4.0.x ================ Requires additional patches for SNAT and FWMARK. Some hurdles with bridge. Bridge problem is to do with packets must be marked PACKET_HOST when heading for br0 as discussed in this tproxy maillist. There have been people saying they will post the patch for it but yet to date, there is none. This problem can be worked around by brouting the traffic into the real devices instead of br0 :- INSIDE_DEV=eth0 OUTSIDE_DEV=eth1 ebtables -t broute -A BROUTING -i $INSIDE_DEV -p ipv4 \ --ip-protocol tcp --ip-destination-port 80 \ -j redirect --redirect-target DROP ebtables -t broute -A BROUTING -i $OUTSIDE_DEV -p ipv4 \ --ip-protocol tcp --ip-source-port 80 \ -j redirect --redirect-target DROP Please note for real interfaces, it's redirect-target DROP and not redirect-target ACCEPT, while doing it on br0, it's redirect-target ACCEPT ! Remember to adjust your iptables rule accordingly since now packets entering and leaving real interfaces instead of br0. Example :- iptables -t tproxy -A PREROUTING -i $INSIDE_DEV \ -p tcp --dport 80 -j TPROXY --on-port 3128 For tproxy-4.0.3 remember to apply the additional kernel patches mentioned in this maillist or else the kernel will panic accessing null pointer. (C) Version tproxy-4.1.0 ================ The ebtables/bridge notes above is equally applicable. However the iptables rules are totally different. Something like this will be required :- iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY \ --tproxy-mark 0x1/0x1 -on-port 3128 iptables -t mangle -N DIVERT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100 _______________________________________________ tproxy mailing list tproxy(a)lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/tproxy

5 16

tproxy-4.0.3-2.6.22[.14] kernel oops
by Gonzalo Arana 29 Feb '08

29 Feb '08

Hi, I am getting a kernel oops with linux kernel 2.6.22.14 & tproxy 4.0.3 when I do "ping -b 255.255.255.255": BUG: unable to handle kernel NULL pointer dereference at virtual address 000000b8 printing eip: c027fc5f *pde = 00000000 Oops: 0000 [#23] PREEMPT SMP Modules linked in: iptable_tproxy ip_tables x_tables vmnet(P) vmblock(P) vmmon(P) i915 drm nfs ipv6 nfsd exportfs lockd sunrpc md_mod dm_snapshot dm_mirror dm_mod mousedev tsdev snd_intel8x0 snd_ac97_codec ac97_bus snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore e100 8139cp 8139too snd_page_alloc 3c59x rtc mii pcspkr i2c_i801 psmouse i2c_core ehci_hcd uhci_hcd intel_agp shpchp pci_hotplug usbcore agpgart evdev trm290 triflex slc90e66 sis5513 siimage serverworks sc1200 rz1000 piix pdc202xx_old pdc202xx_new opti621 ns87415 hpt366 hpt34x generic cy82c693 cs5530 cs5520 cmd64x atiixp amd74xx alim15x3 aec62xx unix via82cxxx ide_generic ide_disk ext3 jbd ide_cd cdrom ide_core floppy CPU: 0 EIP: 0060:[<c027fc5f>] Tainted: P VLI EFLAGS: 00010286 (2.6.22.14 #13) EIP is at __ip_route_output_key+0x20f/0x950 eax: 80000000 ebx: c5681e78 ecx: 490014ac edx: 00000000 esi: 00000000 edi: 00000000 ebp: 00000000 esp: c5681da4 ds: 007b es: 007b fs: 00d8 gs: 0033 ss: 0068 Process ping (pid: 6018, ti=c5680000 task=c5523a70 task.ti=c5680000) Stack: 00000020 0081f2c0 c5681eb4 00000000 00000000 c187eaa8 00000000 00000000 00000001 00000000 ffffffff 490014ac 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 00000000 490314ac 00000000 Call Trace: [<c02803be>] ip_route_output_flow+0x1e/0x250 [<c02a34da>] ip4_datagram_connect+0x15a/0x380 [<c02ad085>] inet_dgram_connect+0x35/0x80 [<c02581e5>] sys_connect+0xd5/0xf0 [<c02d125d>] _spin_unlock+0xd/0x30 [<c0158d94>] __handle_mm_fault+0x224/0x920 [<c02d10dd>] _spin_lock+0xd/0x70 [<c0194ba1>] inotify_d_instantiate+0x21/0x90 [<c02d125d>] _spin_unlock+0xd/0x30 [<c0259dd8>] sys_socketcall+0x98/0x280 [<c0113fe0>] do_page_fault+0x0/0x620 [<c0102b6a>] syscall_call+0x7/0xb ======================= Code: e0 00 00 00 0f 84 c4 03 00 00 81 fa f0 00 00 00 0f 84 5a 02 00 00 84 c0 0f 84 52 02 00 00 8b 44 24 10 8b 54 24 0c 0d 00 00 00 80 <f6> 82 b8 00 00 00 08 0f 44 44 24 10 89 44 24 10 b8 01 00 00 00 EIP: [<c027fc5f>] __ip_route_output_key+0x20f/0x950 SS:ESP 0068:c5681da4 Has anyone faced this problem? Regards, -- Gonzalo A. Arana

3 4

Squid with tproxy extra brief FAQ - take 2
by Ming-Ching Tiew 28 Feb '08

28 Feb '08

1. There are at least 3 different versions of tproxy kernel patches. Each tproxy kernel patch is quite strongly tied to a kernel version, (A) Version Tproxy2 ============= For kernel 2.6.18 URL: http://www.balabit.hu/downloads/files/tproxy/obsolete/ (B) Version Tproxy 4.0.x ================ For kernel 2.6.22 URL: http://www.balabit.hu/downloads/files/tproxy/ (C) Version Tproxy-4.1.0 ================= For kernel 2.6.25 URL: The "official website" is for kernel <=2.6.24 http://people.netfilter.org/hidden/tproxy but the actual version of tproxy 4.1 for 2.6.25 is here: http://people.balabit.hu/panther/tproxy The kernel patch might work with nearby kernel versions, for example, tproxy2 might work with kernel 2.6.19; however it will not work will kernel 2.6.22 ( unless you port it ). 2. Do not confuse tproxy kernel patch mentioned above with squid user-space patches. So far the Squid ( 3.0 and 2.6 ) is only supporting on tproxy2 - the userspace code is integrated. If you managed to compile Squid without changing the source, perhaps with only minor changes in header files, meaning you are likely either did not successfully link in tproxy support or at best it is using tproxy2, and it will not work with tproxy-4.0.x and tproxy-4.1.0 kernel counterpart. However, if you patch the squid source, you should be able to get squid to work with tproxy-4.0.x and tproxy-4.1.0. You can look through the archive of this maillist to look at how to port squid versions to support tproxy-4.0.x and tproxy-4.1.0. Most of the patches floating around are not fully satisfactory, but it could work, at least; but perhaps it will require you to have some programming knowledge. Here maybe a good start :- 3. All the tproxy kernel patches are not compatible with one another. Each requires it's own way of setup and usage. So before doing anything, check if you have gotten the correct info/tproxy version/patches. These are some of the info :- (A) Version Tproxy2 ============ The Squid documentation recommends this :- ebtables -t broute -A BROUTING -p ipv4 --ip-protocol tcp \ --ip-destination-port 80 -j redirect --redirect-target ACCEPT This rule will "broute" bridge traffic from br0 to netfilter. The iptables rule will bring http traffic into local process :- iptables -t tproxy -A PREROUTING -i br0 -p tcp --dport 80 \ -j TPROXY --on-port 3128 To get SNAT working for tproxy2, there is a need for double NAT, and here was the discussion and patch :- https://lists.balabit.hu/pipermail/tproxy/2007-October/000537.html (B) Version tproxy-4.0.x ================ Requires additional patches for SNAT and FWMARK. Some hurdles with bridge. Bridge problem is to do with packets must be marked PACKET_HOST when heading for br0 as discussed in this tproxy maillist. There have been people saying they will post the patch for it but yet to date, there is none. This problem can be worked around by brouting the traffic into the real devices instead of br0 :- INSIDE_DEV=eth0 OUTSIDE_DEV=eth1 ebtables -t broute -A BROUTING -i $INSIDE_DEV -p ipv4 \ --ip-protocol tcp --ip-destination-port 80 \ -j redirect --redirect-target DROP ebtables -t broute -A BROUTING -i $OUTSIDE_DEV -p ipv4 \ --ip-protocol tcp --ip-source-port 80 \ -j redirect --redirect-target DROP Please note for real interfaces, it's redirect-target DROP and not redirect-target ACCEPT, while doing it on br0, it's redirect-target ACCEPT ! Remember to adjust your iptables rule accordingly since now packets entering and leaving real interfaces instead of br0. Example :- iptables -t tproxy -A PREROUTING -i $INSIDE_DEV \ -p tcp --dport 80 -j TPROXY --on-port 3128 For tproxy-4.0.3 remember to apply the additional kernel patches mentioned in this maillist or else the kernel will panic accessing null pointer. (C) Version tproxy-4.1.0 ================ The ebtables/bridge notes above is equally applicable. However the iptables rules are totally different. Something like this will be required :- iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY \ --tproxy-mark 0x1/0x1 -on-port 3128 iptables -t mangle -N DIVERT iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT iptables -t mangle -A DIVERT -j MARK --set-mark 1 iptables -t mangle -A DIVERT -j ACCEPT ip rule add fwmark 1 lookup 100 ip route add local 0.0.0.0/0 dev lo table 100

3 4

squid+tproxy working with slow connection rate (plaintext)
by Auto 27 Feb '08

27 Feb '08

I have squid 2.6.STABLE17 with tproxy compiled in, linux kernel (2.6.20.14) is properly patched with tproxy. Tproxy is working fine and it's spoofing the ip addressed correctly, but there's very strange new connection limit. When i run squid in transparent mode (without tproxy) i got 400-500 conn/sec with httperf and default parms, but with tproxy option i got about 9 conn/sec. Here's the output: httperf --client=0/1 --server=192.168.1.1 --port=65432 --uri=/ --send-buffer=4096 --recv-buffer=16384 --num-conns=1 --num-calls=1 Maximum connect burst length: 0 Total: connections 1 requests 1 replies 1 test-duration 0.110 s Connection rate: 9.1 conn/s (110.2 ms/conn, <=1 concurrent connections) Connection time [ms]: min 110.2 avg 110.2 max 110.2 median 110.5 stddev 0.0 Connection time [ms]: connect 1.8 Connection length [replies/conn]: 1.000 Even on localhost when i get about 4000 conn/sec i got only 9 with tproxy option. Is there something that limits the number of new connections and delays SYN ?

2 1

TProxy installation
by Eduardo Schoedler 26 Feb '08

26 Feb '08

Hi all. I'm Brazilian, so sorry my bad english. =) I'm looking for a installation manual, with _ALL_ the steps necessary to install the tproxy patch. I found this link, but isn't working. http://fuzzylab00net.blog.dada.net/post/413913/Squid-2.6-+-tproxy-+-bridge-… I'm trying build an linux box with 2 nic's, in a bridge setup. I have an "real" IP in my bridge interface (189.x.x.x). I have built a lot of kernels with a lot of tproxy's patches. Here is the kernel output: # cat /var/log/dmesg | grep -i tproxy IP_TPROXY: Transparent proxy support initialized, version 4.0.0 IP_TPROXY: Copyright (c) 2002-2007 BalaBit IT Ltd. I have used the file tproxy-4.0.3-2.6.22.tgz to patch my kernel (2.6.22-gentoo-r10). The same patch I've used to compile my iptables-1.3.8. My squid is compiled with the patch too, and my squid.conf is configured: http_port 3128 tproxy transparent tcp_outgoing_address 189.x.x.x My kernel is also configured: net.ipv4.ip_nonlocal_bind = 1 net.ipv4.ip_forward = 1 My ebtables and iptables rules: ebtables -t broute -A BROUTING -p IPv4 --ip-protocol 6 --ip-destination-port 80 -j redirect --redirect-target ACCEPT iptables -t tproxy -A PREROUTING -i br0 -p tcp --dport 80 -j TPROXY --on-port 3128 What I do wrong ? Thanks! Best Regards, Eduardo Schoedler.

1 0

squid+tproxy working with slow connection rate
by Auto 26 Feb '08

26 Feb '08

1 0

problem with tproxy and iptables
by nantenaina Tianarivo 25 Feb '08

25 Feb '08

Hello everybody, I am tring to make tproxy work with our squid but I have a problem with the iptable to redirect traffic to squid now. I have compiled a linux kernel 2.6.22.18 patched with tproxy-4.0.3-2.6.22. and iptable 1.3.8. For squid, i'am using Version 2.6.STABLE5. I think my kernel is well compiled because I see all the tproxy module loaded : proxy:/usr/src/linux# lsmod | grep -i proxy xt_tproxy 1984 0 xt_TPROXY 1984 1 iptable_tproxy 6468 2 xt_TPROXY ip_tables 12420 2 iptable_filter,iptable_tproxy x_tables 14564 5 ipt_LOG,xt_tcpudp,xt_tproxy,xt_TPROXY,ip_tables My iptables rules is like this : iptables -t tproxy -A PREROUTING -p tcp -m tcp -i gre1 --dport 80 -j LOG iptables -t tproxy -A PREROUTING -p tcp -m tcp -i gre1 --dport 80 -j TPROXY --on-port 80 when I check it with tcpdump, I see traffic for http port on the gre1 interface proxy:/usr/src/linux# tcpdump -n -i gre1 tcpdump: WARNING: arptype 778 not supported by libpcap - falling back to cooked socket tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on gre1, link-type LINUX_SLL (Linux cooked), capture size 96 bytes 13:10:51.437856 IP 62.56.240.17.3200 > 84.16.80.10.80: . ack 3247536657 win 2264 <nop,nop,timestamp 24199037 1582152> 13:10:51.492666 IP 62.56.240.17.3199 > 84.16.80.10.80: . ack 3204902926 win 3604 <nop,nop,timestamp 24199051 1582156> 13:10:51.523999 IP 62.56.240.17.3198 > 84.16.80.10.80: . ack 3189913679 win 16022 <nop,nop,timestamp 24199058 1582173> when I check it on access.log of my squid, my requests are actually sent to the squid. But it is not the client ip which is sent to the Internet but the squid box IP. when I issue iptables-save -c command to check if there are traffic that enter my iptables rule, the counter so zero traffic. proxy:/usr/src/linux# iptables-save -t tproxy -c # Generated by iptables-save v1.3.8 on Wed Feb 20 13:07:45 2008 *tproxy :PREROUTING ACCEPT [128:11992] [0:0] -A PREROUTING -i gre1 -p tcp -m tcp --dport 80 -j LOG [0:0] -A PREROUTING -i gre1 -p tcp -m tcp --dport 80 -j TPROXY --on-port 80 --on-ip 0.0.0.0 COMMIT even the LOG don't tell me anything about traffic in gre1 interfaces. what I see in the log is this error that appear from time to time: Feb 20 13:08:31 proxy squid[2353]: parseHttpRequest: NF getsockopt(SO_ORIGINAL_DST) failed: (92) Protocol not available Feb 20 13:08:31 proxy squid[2353]: tproxy ip=62.56.240.17,0x11f0383e,port=0 ERROR ASSIGN Can anyone tell me why my iptables don't redirect the traffic to the squid? Best Regards Nantenaina Tianarivo Sys Admin Blueline ------------------- Today is what happened to yesterday.

4 8

tproxy and squid-3.0.STABLE1-20080220
by Remzi AKYUZ 20 Feb '08

20 Feb '08

Hello, I want to try squid 3 and tproxy. ./configure --prefix=/home/squid3/tproxy-test --with-filedescriptors=16384 --enable-x-accelerator-vary --enable-linux-tproxy --enable-linux-netfilter ... checking if Linux (Netfilter) kernel header files are installed... yes checking if TPROXY header files are installed... no WARNING: Cannot find TPROXY headers, you need to install the tproxy package from: - lynx http://www.balabit.com/downloads/tproxy/ ... I installed tproxy-4.0.3-2.6.22 to kernel and iptables. how can install tproxy headers files? Thank for your time. Remzi

2 1

Does TPROXY work on a bridge without IP address?
by Liang Bin 20 Feb '08

20 Feb '08

Hi all, I am Bryan Liang, a newbie to TPROXY. I am using TPROXY 2.0.6 with kernel 2.6.20.21. It works fine in router mode. I am going to run TPROXY in bridge mode. Here is my linux box setting. eth0: local admin port, IP address: 192.168.43.130, netmask 255.255.255.0, default gw 192.168.43.1 eth1 and eth2 belong to br0 (bridge device) br0: IP address: 0.0.0.0 This means I can use eth0 as admin port to manage my linux box. And I do not assign IP address to bridge interface. My problem is, does TPROXY work on a bridge without IP address? Thank you in advance. :) Thanks, -Bryan

3 3

tproxy 4.1.0 and FWMARK
by Ming-Ching Tiew 19 Feb '08

19 Feb '08

OK I could confirm that the SNAT patch provided seems to work for me. I have more questions on tproxy 4.1.0 and FWMARK :- > iptables -t mangle -A PREROUTING -p tcp --dport 80 -j TPROXY \ > --tproxy-mark 0x1/0x1 -on-port 3128 > iptables -t mangle -N DIVERT > iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT > iptables -t mangle -A DIVERT -j MARK --set-mark 1 > iptables -t mangle -A DIVERT -j ACCEPT > > ip rule add fwmark 1 lookup 100 > ip route add local 0.0.0.0/0 dev lo table 100 If I have to do split access on outgoing traffic into multiple interfaces, I would already have use FWMARK in one way or other. For example, for split outgoing access, I would have already mark traffic going out to eth0 with FWMARK 5, and traffic going out to eth1 with FWMARK 7. And I would already have these rules :- ip rule add fwmark 5 lookup 120 ip rule add fwmark 7 lookup 130 ( And table 120 will default route to gateway belonging to interface 1 and table 130 will default route to gateway belonging to interface 2 ). So how could I incorporate the tproxy mark into my existing use of FWMARK ? Regards.

2 1