April 2004 - tproxy - lists.balabit.hu

RE: [tproxy] cttproxy-2.4.25-1.9.3 + nat_delete
by Andrew Ivins 27 Apr '04

27 Apr '04

Thanks Krisztian, you were spot on :) The test program can now connect repeatedly without problems. Thanks also for updating your patch, although I'll hold off on using nat_delete as you suggest. Regards, Andrew -----Original Message----- From: KOVACS Krisztian [mailto:hidden@balabit.hu] Sent: Tuesday, 27 April 2004 4:01 PM To: Andrew Ivins Cc: TProxy mailing list Subject: Re: [tproxy] cttproxy-2.4.25-1.9.3 + nat_delete Hi, On Tue, 2004-04-27 at 09:22, Andrew Ivins wrote: > I have experimented with binding to a foreign source address using > cttproxy-2.4.25-1.9.3 and the foreign-tcp-connect program. However I > skipped the nat_delete part of the patch as I was not able to apply > it. Foreign-tcp-connect actually works perfectly with the peer seeing > the spoofed address. However after working, it will consistently fail > for several minutes. Then it will work once and the cycle will repeat. > I'm guessing this has something to do with the nat_delete patch I > skipped. Actually, 04-nat_delete is not mandatory at all (although could help in your case). The root of your problems is probably that you set the foreign address and port number to the same constant value all the time, and connect to the same server. In this case, when the first TCP session is closed, you'll have to wait two minutes for the conntrack entry to time out. Instead of using 04-nat_delete to be able to delete leftover conntrack entries of TCP connections currently in TIME_WAIT state, I'd recommend not to use constant values when using tproxy for connecting. If you specify 0 for the foreign port, an unused port number will be automatically allocated by Netfilter, and you won't get NAT clashes. The 04-* patch is needed in such cases when this is not an option, which is very rare. > > The nat_delete patch has been adapted to the new TCP window-tracking > > patch by Jozsef Kadlecsik, so this patch needs the current > > tcp-window-tracking module from the Netfilter Patch-o-matic NG > applied. > > If you don't want do do that, 04-nat_delete.patch can be skipped > safely. > > I tried this. The tcp-window-tracking patch seemed to apply to the > vanilla 2.4.25 kernel successfully. However the 04-nat_delete.patch > still fails to apply (included below) > > Any chance of a list of the which patches need to be applied to make > cttproxy-2.4.25-1.9.3 apply cleanly including 4-nat_delete.patch? You're right. Although I wouldn't recommend using that patch, I've created a fixed archive which contains the new 04-nat_delete.diff and replaced the old .tar.gz. The MD5 checksum of the new file is 64eb5a4e72f11fa98d7794ed0bb8ee2a cttproxy-2.4.25-1.9.3.tar.gz -- Regards, Krisztian KOVACS

1 0

cttproxy-2.4.25-1.9.3 + nat_delete
by Andrew Ivins 27 Apr '04

27 Apr '04

Hi, this may be a question for KOVACS Krisztian: I have experimented with binding to a foreign source address using cttproxy-2.4.25-1.9.3 and the foreign-tcp-connect program. However I skipped the nat_delete part of the patch as I was not able to apply it. Foreign-tcp-connect actually works perfectly with the peer seeing the spoofed address. However after working, it will consistently fail for several minutes. Then it will work once and the cycle will repeat. I'm guessing this has something to do with the nat_delete patch I skipped. From the mailing list archives: > The nat_delete patch has been adapted to the new TCP window-tracking > patch by Jozsef Kadlecsik, so this patch needs the current > tcp-window-tracking module from the Netfilter Patch-o-matic NG applied. > If you don't want do do that, 04-nat_delete.patch can be skipped safely. I tried this. The tcp-window-tracking patch seemed to apply to the vanilla 2.4.25 kernel successfully. However the 04-nat_delete.patch still fails to apply (included below) Any chance of a list of the which patches need to be applied to make cttproxy-2.4.25-1.9.3 apply cleanly including 4-nat_delete.patch? Regards, Andrew /usr/src/linux# patch -p1 < ../cttproxy-2.4.25-1.9.3/patch_tree/04-nat_delete.diff patching file net/ipv4/netfilter/ip_conntrack_core.c Hunk #1 FAILED at 357. 1 out of 4 hunks FAILED -- saving rejects to file net/ipv4/netfilter/ip_conntrack_core.c.rej patching file net/ipv4/netfilter/ip_nat_core.c Hunk #2 succeeded at 285 with fuzz 1. patching file net/ipv4/netfilter/ip_conntrack_proto_tcp.c patching file net/ipv4/netfilter/ip_conntrack_standalone.c Hunk #1 FAILED at 508. 1 out of 1 hunk FAILED -- saving rejects to file net/ipv4/netfilter/ip_conntrack_standalone.c.rej patching file include/linux/netfilter_ipv4/ip_conntrack.h Hunk #3 FAILED at 263. 1 out of 3 hunks FAILED -- saving rejects to file include/linux/netfilter_ipv4/ip_conntrack.h.rej

2 1

Re: [tproxy] Patch errors on vanilla kernels?
by David 24 Apr '04

24 Apr '04

(I accidently sent this mail to Balazs earlier today. Was checking the message, and found out I didn't send it to the mailinglist. Sorry to Balazs for that) "for example ..." Do you use any other patch or is this tcp-window-tracking patch the only one? And shouldn't dependencies be in the README file? I have no clue which patches to install before I can use the TProxy patches. David Balazs Scheidler said: >> Applying the patches on my system fails. I've tried several different versions: >> cttproxy-2.4.25-1.2.0 on a vanilla Linux 2.4.25 kernel >> cttproxy-2.4.25-1.9.3 on a vanilla Linux 2.4.25 kernel >> cttproxy-2.6.3-1.9.2 on a vanilla Linux 2.6.3 kernel >> cttproxy-2.6.5-1.9.3 on a vanilla Linux 2.6.3 kernel >> All patches fail. The best option is cttproxy-2.4.25-1.2.0, which only generates one error, that may possibly be fixed by hand (ip_contrack.h). The worst option is cttproxy-2.6.5-1.9.3, which generates dozens of errors. >> >> Do you use pre-patched (red hat, suse) kernels or something? I don't know >> what else could have caused these problems. Does anyone know this problem? > > no, we are using vanilla kernels, but tproxy also has a couple of dependencies. (for example the latest 1.9.3 releases need > tcp-window-tracking from Jozsef Kadlecsik)

2 1

failure to apply snat mapping?
by Lennert Buytenhek 23 Apr '04

23 Apr '04

Hi, Let's say that I have an app that connects to IP address 1.2.3.4, and uses tproxy to fake the source address as 5.6.7.8. Bind the socket, then call into tproxy, and then connect() and ta da -- everything works as expected. Now I decide that that app should not connect to 1.2.3.4, but instead to 1.2.3.5. I don't want to modify the source and restart it, so I add a nat rule in the iptables nat/OUTPUT chain to DNAT the address to 1.2.3.5. The app now (unknowingly) connects to 1.2.3.5, that works fine. But.. the source address used for the connection is now the source address of the box and not anymore 5.6.7.8? :(( Is this a case of "Don't do that, then!!"? I'm using "tproxy-2.4.22-1.1.3.diff" patched into a Red Hat 2.4.20 kernel (2.4.20-20.9 to be exact) on a uniproc P4 2.4GHz, 1G RAM. cheers, Lennert

2 3

Patch errors on vanilla kernels?
by David 22 Apr '04

22 Apr '04

Hi, Applying the patches on my system fails. I've tried several different versions: cttproxy-2.4.25-1.2.0 on a vanilla Linux 2.4.25 kernel cttproxy-2.4.25-1.9.3 on a vanilla Linux 2.4.25 kernel cttproxy-2.6.3-1.9.2 on a vanilla Linux 2.6.3 kernel cttproxy-2.6.5-1.9.3 on a vanilla Linux 2.6.3 kernel All patches fail. The best option is cttproxy-2.4.25-1.2.0, which only generates one error, that may possibly be fixed by hand (ip_contrack.h). The worst option is cttproxy-2.6.5-1.9.3, which generates dozens of errors. Do you use pre-patched (red hat, suse) kernels or something? I don't know what else could have caused these problems. Does anyone know this problem?

2 1

dynamic proxying?
by Qiang zhang 16 Apr '04

16 Apr '04

Hello, List I hope this question is not bothering, Can i use the tproxy with iptables to make a dynamic proxying? by this I mean if an incoming connection is seen not going anywhere (i.e. there is no service/server available), I want to add a proxy rule dynamically so the connection will be redirect to another fixed IP Is this at all do-able by using TPROXY? Please shed some idea, thanks a bunch! Q __________________________________ Do you Yahoo!? Yahoo! Tax Center - File online by April 15th http://taxes.yahoo.com/filing.html

1 0

Development version 1.9.3 released
by KOVACS Krisztian 07 Apr '04

07 Apr '04

Hi, A new version of the current TProxy development branch has been released. Support of Linux 2.4 is back, although through two completely indenpendent patch sets. There were a couple of compilation problems with 1.9.2, and a few ugly initialization bugs, which caused the iptable_tproxy module to cause oopses at load. With the help of James MacLean, these bugs seem to be solved. (For more information, see ChangeLog.) The nat_delete patch has been adapted to the new TCP window-tracking patch by Jozsef Kadlecsik, so this patch needs the current tcp-window-tracking module from the Netfilter Patch-o-matic NG applied. If you don't want do do that, 04-nat_delete.patch can be skipped safely. The documentation is still not up-to-date, unfortunately. The release can be downloaded from http://www.balabit.com/downloads/tproxy/linux-2.4/devel/ MD5: cttproxy-2.4.25-1.9.3.tar.gz eb3b5a4880a68c87e5340643b171ee20 MD5: cttproxy-2.6.5-1.9.3.tar.gz d64c3bcaae81f08b804ce910dbdb752a -- Regards, Krisztian KOVACS

1 0

Re: [tproxy] anything known about potential memory leaks in tproxy 1.1.3 ?
by Lennert Buytenhek 06 Apr '04

06 Apr '04

On Tue, Apr 06, 2004 at 11:50:09AM +0200, KOVACS Krisztian wrote: > Hi, Hi, > > Our production firewall/transproxy suddenly became _really_ unresponsive > > a few days ago after ~120 days of uptime, from what seems to have been a > > memory leak. > > > > The machine has 1GB of memory, but the amount of page cache was really small, > > the number of processes waiting for disk I/O was about 30, and at avg 4MB/sec > > reading from disk it was seeking all over the place. Also, bash would > > sometimes say something along the lines of 'resource unavailable' when > > starting subprocesses. > > > > Temporarily disabling the transproxy function by deleting the redirect rule > > caused the disk to go idle (our transproxy-using app intercepts HTTP requests > > and answers the requests locally for static data, fileset is about 5GB but > > ~200MB of it accounts for 90% of the hits.) But every little thing that I > > tried, even an ls or a grep in /etc caused massive disk I/O. > > > > I hastily rebooted the machine and thus have no ways of debugging this > > anymore. But I was wondering, is anything known about memory leaks in the > > 1.1.3 version of the tproxy code? I checked the changelogs, but couldn't > > find anything related to memleaks. > > > > Should I schedule maintenance to move the machine to 1.9.1? Any other > > advice? (It's not a problem for me to schedule the machine to be rebooted > > every 90 days or something like that.) > > We've also found signs that suggest that there may be a possible > memory leak in tproxy (it seemed to leak conntrack entries). However, I > don't have an idea what the problem might be, so it's not fixed yet. > > Could you provide a bit more information? Is this system SMP or UP? > I'd be happy to receive the contents of /proc/slabinfo, if possible, > daily/hourly snapshots. Uniprocessor system, P4 2.4GHz. Two e1000 fiber cards, 2.4.20-20.9 Red Hat kernel plus bridging firewall patch and tproxy patch 1.1.3. Right now about 200 SYNs per second, which is sort-of a quiet time of day. /proc/net/ip_conntrack right now shows about 35k entries, 'netstat -tn' shows about 7k. I've installed a cron job to take regular snapshots of /proc/slabinfo, will get back to you soon. > Moving to 1.9.1 is not recommended in your case, its ABI/API is > incompatible (though trivial to fix in your sources), and probably won't > help. OK. Thanks so far. cheers, Lennert

1 0

anything known about potential memory leaks in tproxy 1.1.3 ?
by Lennert Buytenhek 06 Apr '04

06 Apr '04

Hi, Our production firewall/transproxy suddenly became _really_ unresponsive a few days ago after ~120 days of uptime, from what seems to have been a memory leak. The machine has 1GB of memory, but the amount of page cache was really small, the number of processes waiting for disk I/O was about 30, and at avg 4MB/sec reading from disk it was seeking all over the place. Also, bash would sometimes say something along the lines of 'resource unavailable' when starting subprocesses. Temporarily disabling the transproxy function by deleting the redirect rule caused the disk to go idle (our transproxy-using app intercepts HTTP requests and answers the requests locally for static data, fileset is about 5GB but ~200MB of it accounts for 90% of the hits.) But every little thing that I tried, even an ls or a grep in /etc caused massive disk I/O. I hastily rebooted the machine and thus have no ways of debugging this anymore. But I was wondering, is anything known about memory leaks in the 1.1.3 version of the tproxy code? I checked the changelogs, but couldn't find anything related to memleaks. Should I schedule maintenance to move the machine to 1.9.1? Any other advice? (It's not a problem for me to schedule the machine to be rebooted every 90 days or something like that.) cheers, Lennert

2 2