Hi, TProxy version 4.0.0 has been released for Linux 2.6.17 (Ubuntu 2.6.17.1-12.39) and 2.6.23-rc1 This is a completely rewritten version which no longer uses NAT. The release tarballs are available here: http://www.balabit.com/downloads/files/tproxy/ MD5 checksums of the tarballs: a655fd090c06827bd6927f2b894ee178 tproxy-4.0.0-2.6.23-rc1.tgz d7175531d67beaab48293cd86cac5d21 tproxy-4.0.0-ubuntu-2.6.17-12.39.tgz -- Regards, Laszlo Attila Toth
Looks like I should join in my modifications. === iptables TPROXY target From: KOVACS Krisztian <hidden@balabit.hu> The TPROXY target implements redirection of non-local TCP/UDP traffic to local sockets. It is simply a wrapper around functionality exported from iptable_tproxy. Signed-off-by: KOVACS Krisztian <hidden@balabit.hu> Changed to xt_TPROXY. -Jan Engelhardt <jengelh@gmx.de> --- include/linux/netfilter/xt_TPROXY.h | 9 ++++ net/ipv4/netfilter/Kconfig | 11 +++++ net/netfilter/Makefile | 1 net/netfilter/xt_TPROXY.c | 76 ++++++++++++++++++++++++++++++++++++ 4 files changed, 97 insertions(+) Index: linux-2.6.22.1/include/linux/netfilter/xt_TPROXY.h =================================================================== --- /dev/null +++ linux-2.6.22.1/include/linux/netfilter/xt_TPROXY.h @@ -0,0 +1,9 @@ +#ifndef _XT_TPROXY_H +#define _XT_TPROXY_H + +struct xt_tproxy_info { + u_int32_t laddr; + u_int16_t lport; +}; + +#endif /* _XT_TPROXY_H */ Index: linux-2.6.22.1/net/ipv4/netfilter/Kconfig =================================================================== --- linux-2.6.22.1.orig/net/ipv4/netfilter/Kconfig +++ linux-2.6.22.1/net/ipv4/netfilter/Kconfig @@ -403,6 +403,17 @@ config IP_NF_TPROXY_TABLE To compile it as a module, choose M here. If unsure, say `N'. +config NETFILTER_XT_TARGET_TPROXY + tristate "TPROXY target support" + depends on IP_NF_TPROXY_TABLE + help + This option adds a `TPROXY' target, which is somewhat similar to + REDIRECT. It can only be used in the tproxy table and is useful + to redirect traffic to a transparent proxy. It does _not_ depend + on Netfilter connection tracking. + + To compile it as a module, choose M here. If unsure, say N. + # ARP tables config IP_NF_ARPTABLES tristate "ARP tables support" Index: linux-2.6.22.1/net/netfilter/Makefile =================================================================== --- linux-2.6.22.1.orig/net/netfilter/Makefile +++ linux-2.6.22.1/net/netfilter/Makefile @@ -50,6 +50,7 @@ obj-$(CONFIG_NETFILTER_XT_TARGET_TRACE) obj-$(CONFIG_NETFILTER_XT_TARGET_SECMARK) += xt_SECMARK.o obj-$(CONFIG_NETFILTER_XT_TARGET_TARPIT) += xt_TARPIT.o obj-$(CONFIG_NETFILTER_XT_TARGET_TCPMSS) += xt_TCPMSS.o +obj-$(CONFIG_NETFILTER_XT_TARGET_TPROXY) += xt_TPROXY.o obj-$(CONFIG_NETFILTER_XT_TARGET_CONNSECMARK) += xt_CONNSECMARK.o # matches Index: linux-2.6.22.1/net/netfilter/xt_TPROXY.c =================================================================== --- /dev/null +++ linux-2.6.22.1/net/netfilter/xt_TPROXY.c @@ -0,0 +1,76 @@ +/* + * Transparent proxy support for Linux/iptables + * + * Copyright (c) 2006-2007 BalaBit IT Ltd. + * Author: Balazs Scheidler, Krisztian Kovacs + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ +#include <linux/ip.h> +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/skbuff.h> +#include <linux/netfilter_ipv4/ip_tables.h> +#include <linux/netfilter_ipv4/ip_tproxy.h> +#include <linux/netfilter/xt_TPROXY.h> +#include <net/checksum.h> +#include <net/inet_sock.h> +#include <net/udp.h> + +static unsigned int +tproxy_target(struct sk_buff **pskb, const struct net_device *in, + const struct net_device *out, unsigned int hooknum, + const struct xt_target *target, const void *targinfo) +{ + const struct xt_tproxy_info *tgi = targinfo; + const struct iphdr *iph = ip_hdr(*pskb); + struct sk_buff *skb = *pskb; + struct udphdr _hdr, *hp; + + /* TCP/UDP only */ + if (iph->protocol != IPPROTO_TCP && iph->protocol != IPPROTO_UDP) + return NF_ACCEPT; + + hp = skb_header_pointer(*pskb, iph->ihl * 4, sizeof(_hdr), &_hdr); + if (hp == NULL) + return NF_DROP; + + skb->nf_tproxy.redirect_address = tgi->laddr ? : iph->daddr; + skb->nf_tproxy.redirect_port = tgi->lport ? : hp->dest; + + pr_debug(KERN_DEBUG "redirecting: proto %d %08x:%d -> %08x:%d\n", + iph->protocol, ntohl(iph->daddr), ntohs(hp->dest), + ntohl(skb->nf_tproxy.redirect_address), + ntohs(skb->nf_tproxy.redirect_port)); + + return NF_ACCEPT; +} + +static struct xt_target xt_tproxy_reg __read_mostly = { + .name = "TPROXY", + .family = AF_INET, + .table = "tproxy", + .target = tproxy_target, + .targetsize = sizeof(struct xt_tproxy_info), + .me = THIS_MODULE, +}; + +static int __init xt_tproxy_init(void) +{ + return xt_register_target(&xt_tproxy_reg); +} + +static void __exit xt_tproxy_exit(void) +{ + xt_unregister_target(&xt_tproxy_reg); +} + +module_init(xt_tproxy_init); +module_exit(xt_tproxy_exit); +MODULE_AUTHOR("Krisztian Kovacs <hidden@balabit.hu>"); +MODULE_DESCRIPTION("Netfilter transparent proxy TPROXY target module"); +MODULE_LICENSE("GPL"); +MODULE_ALIAS("ipt_TPROXY");
iptables socket match From: KOVACS Krisztian <hidden@balabit.hu> Add iptables 'socket' match, which matches packets for which a TCP/UDP socket lookup succeeds. Signed-off-by: KOVACS Krisztian <hidden@balabit.hu> Changed to xt_socket. -Jan Engelhardt <jengelh@gmx.de> --- net/ipv4/netfilter/Kconfig | 10 +++++ net/netfilter/Makefile | 1 net/netfilter/xt_socket.c | 80 +++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 91 insertions(+) Index: linux-2.6.22.1/net/ipv4/netfilter/Kconfig =================================================================== --- linux-2.6.22.1.orig/net/ipv4/netfilter/Kconfig +++ linux-2.6.22.1/net/ipv4/netfilter/Kconfig @@ -414,6 +414,16 @@ config NETFILTER_XT_TARGET_TPROXY To compile it as a module, choose M here. If unsure, say N. +config NETFILTER_XT_MATCH_SOCKET + tristate "socket match support" + depends on IP_NF_TPROXY_TABLE + help + This option adds a `socket' match, which can be used to match + packets for which a TCP or UDP socket lookup finds a valid socket. + It can only be used in the tproxy table. + + To compile it as a module, choose M here. If unsure, say N. + # ARP tables config IP_NF_ARPTABLES tristate "ARP tables support" Index: linux-2.6.22.1/net/netfilter/Makefile =================================================================== --- linux-2.6.22.1.orig/net/netfilter/Makefile +++ linux-2.6.22.1/net/netfilter/Makefile @@ -76,6 +76,7 @@ obj-$(CONFIG_NETFILTER_XT_MATCH_PKTTYPE) obj-$(CONFIG_NETFILTER_XT_MATCH_QUOTA) += xt_quota.o obj-$(CONFIG_NETFILTER_XT_MATCH_REALM) += xt_realm.o obj-$(CONFIG_NETFILTER_XT_MATCH_SCTP) += xt_sctp.o +obj-$(CONFIG_NETFILTER_XT_MATCH_SOCKET) += xt_socket.o obj-$(CONFIG_NETFILTER_XT_MATCH_STATE) += xt_state.o obj-$(CONFIG_NETFILTER_XT_MATCH_STATISTIC) += xt_statistic.o obj-$(CONFIG_NETFILTER_XT_MATCH_STRING) += xt_string.o Index: linux-2.6.22.1/net/netfilter/xt_socket.c =================================================================== --- /dev/null +++ linux-2.6.22.1/net/netfilter/xt_socket.c @@ -0,0 +1,80 @@ +/* + * Transparent proxy support for Linux/iptables + * + * Copyright (c) 2007 BalaBit IT Ltd. + * Author: Krisztian Kovacs + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2 as + * published by the Free Software Foundation. + * + */ +#include <linux/kernel.h> +#include <linux/module.h> +#include <linux/skbuff.h> +#include <linux/netfilter/x_tables.h> +#include <linux/netfilter_ipv4/ip_tproxy.h> +#include <net/inet_sock.h> +#include <net/sock.h> +#include <net/tcp.h> +#include <net/udp.h> + +static int +socket_match(const struct sk_buff *skb, const struct net_device *in, + const struct net_device *out, const struct xt_match *match, + const void *matchinfo, int offset, unsigned int protoff, + int *hotdrop) +{ + const struct iphdr *iph = ip_hdr(skb); + struct udphdr _hdr, *hp; + struct sock *sk; + + /* TCP/UDP only */ + if (iph->protocol != IPPROTO_TCP && iph->protocol != IPPROTO_UDP) + return false; + + hp = skb_header_pointer(skb, iph->ihl * 4, sizeof(_hdr), &_hdr); + if (hp == NULL) + return false; + + sk = ip_tproxy_get_sock(iph->protocol, iph->saddr, iph->daddr, + hp->source, hp->dest, in); + if (sk != NULL) { + if (iph->protocol == IPPROTO_TCP && + sk->sk_state == TCP_TIME_WAIT) + inet_twsk_put(inet_twsk(sk)); + else + sock_put(sk); + } + + pr_debug(KERN_DEBUG "socket match: proto %d %08x:%d -> %08x:%d sock %p\n", + iph->protocol, ntohl(iph->saddr), ntohs(hp->source), + ntohl(iph->daddr), ntohs(hp->dest), sk); + + return sk != NULL; +} + +static struct xt_match socket_reg __read_mostly = { + .name = "socket", + .family = AF_INET, + .table = "tproxy", + .match = socket_match, + .me = THIS_MODULE, +}; + +static int __init xt_socket_init(void) +{ + return xt_register_match(&socket_reg); +} + +static void __exit xt_socket_fini(void) +{ + xt_unregister_match(&socket_reg); +} + +module_init(xt_socket_init); +module_exit(xt_socket_fini); +MODULE_AUTHOR("Krisztian Kovacs <hidden@balabit.hu>"); +MODULE_DESCRIPTION("netfilter socket match module"); +MODULE_LICENSE("GPL"); +MODULE_ALIAS("ipt_socket");
Previous two were for 2.6.22, if using nf-dev, the following extra is needed. --- net/netfilter/xt_socket.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) Index: linux-2.6.22.1/net/netfilter/xt_socket.c =================================================================== --- linux-2.6.22.1.orig/net/netfilter/xt_socket.c +++ linux-2.6.22.1/net/netfilter/xt_socket.c @@ -19,11 +19,11 @@ #include <net/tcp.h> #include <net/udp.h> -static int +static bool socket_match(const struct sk_buff *skb, const struct net_device *in, const struct net_device *out, const struct xt_match *match, const void *matchinfo, int offset, unsigned int protoff, - int *hotdrop) + bool *hotdrop) { const struct iphdr *iph = ip_hdr(skb); struct udphdr _hdr, *hp;
Applies on top of iptables-svn6974. --- extensions/.tproxy-testx | 3 + extensions/libxt_TPROXY.c | 114 ++++++++++++++++++++++++++++++++++++++++++++++ extensions/libxt_socket.c | 49 +++++++++++++++++++ 3 files changed, 166 insertions(+) Index: iptables/extensions/.tproxy-testx =================================================================== --- /dev/null +++ iptables/extensions/.tproxy-testx @@ -0,0 +1,3 @@ +#!/bin/sh +[ -f "$KERNEL_DIR/include/linux/netfilter/xt_TPROXY.h" ] && echo TPROXY; +echo socket; Index: iptables/extensions/libxt_TPROXY.c =================================================================== --- /dev/null +++ iptables/extensions/libxt_TPROXY.c @@ -0,0 +1,114 @@ +/* Shared library add-on to iptables to add TPROXY target support. + * + * Copyright (C) 2002-2007 BalaBit IT Ltd. + */ +#include <stdio.h> +#include <string.h> +#include <stdlib.h> +#include <getopt.h> + +#include <iptables.h> +#include <xtables.h> +#include <linux/netfilter_ipv4/ip_tables.h> +#include <linux/netfilter/xt_TPROXY.h> + +static const struct option tproxy_opts[] = { + {"on-port", 1, NULL, '1'}, + {"on-ip", 1, NULL, '2'}, + {NULL}, +}; + +static void tproxy_help(void) +{ + printf( +"TPROXY target v%s options:\n" +" --on-port port Redirect connection to port, or the original port if 0\n" +" --on-ip ip Optionally redirect to the given IP\n", +IPTABLES_VERSION); +} + +static void parse_tproxy_lport(const char *s, struct xt_tproxy_info *info) +{ + unsigned int lport; + + if (string_to_number(s, 0, 65535, &lport) != -1) + info->lport = htons(lport); + else + exit_error(PARAMETER_PROBLEM, "bad --on-proxy `%s'", s); +} + +static void parse_tproxy_laddr(const char *s, struct xt_tproxy_info *info) +{ + struct in_addr *laddr; + + if ((laddr = dotted_to_addr(s)) == NULL) + exit_error(PARAMETER_PROBLEM, "bad --on-ip `%s'", s); + info->laddr = laddr->s_addr; +} + +static int tproxy_parse(int c, char **argv, int invert, unsigned int *flags, + const void *entry, struct xt_entry_target **target) +{ + struct xt_tproxy_info *tproxyinfo = (void *)(*target)->data; + + switch (c) { + case '1': + if (*flags != 0) + exit_error(PARAMETER_PROBLEM, + "TPROXY target: Can't specify --to-port twice"); + parse_tproxy_lport(optarg, tproxyinfo); + *flags = 1; + break; + case '2': + parse_tproxy_laddr(optarg, tproxyinfo); + break; + default: + return 0; + } + + return 1; +} + +static void tproxy_check(unsigned int flags) +{ + if (flags == 0) + exit_error(PARAMETER_PROBLEM, + "TPROXY target: Parameter --on-port is required"); +} + +static void tproxy_print(const void *ip, const struct xt_entry_target *target, + int numeric) +{ + const struct xt_tproxy_info *tproxyinfo = (const void *)target->data; + printf("TPROXY redirect %s:%d", + addr_to_dotted((const struct in_addr *)&tproxyinfo->laddr), + ntohs(tproxyinfo->lport)); +} + +static void tproxy_save(const void *ip, const struct xt_entry_target *target) +{ + const struct xt_tproxy_info *tproxyinfo = (const void *)target->data; + + printf("--on-port %d ", ntohs(tproxyinfo->lport)); + printf("--on-ip %s ", + addr_to_dotted((const struct in_addr *)&tproxyinfo->laddr)); +} + +static struct xtables_target tproxy_reg = { + .name = "TPROXY", + .family = AF_INET, + .version = IPTABLES_VERSION, + .size = XT_ALIGN(sizeof(struct xt_tproxy_info)), + .userspacesize = XT_ALIGN(sizeof(struct xt_tproxy_info)), + .help = tproxy_help, + .parse = tproxy_parse, + .final_check = tproxy_check, + .print = tproxy_print, + .save = tproxy_save, + .extra_opts = tproxy_opts, +}; + +void _init(void) +{ + xtables_register_target(&tproxy_reg); +} Index: iptables/extensions/libxt_socket.c =================================================================== --- /dev/null +++ iptables/extensions/libxt_socket.c @@ -0,0 +1,49 @@ +/* Shared library add-on to iptables to add early socket matching support. */ +#include <stdio.h> +#include <getopt.h> +#include <xtables.h> + +static void socket_print(const void *ip, const struct xt_entry_match *match, + int numeric) +{ + printf("socket "); +} + +static int socket_parse(int c, char **argv, int invert, unsigned int *flags, + const void *entry, unsigned int *nfcache, + struct xt_entry_match **match) +{ + return 0; +} + +static void socket_check(unsigned int flags) +{ +} + +static struct xtables_match socket_reg = { + .name = "socket", + .family = AF_INET, + .version = IPTABLES_VERSION, + .size = XT_ALIGN(0), + .userspacesize = XT_ALIGN(0), + .parse = socket_parse, + .final_check = socket_check, + .print = socket_print, +}; + +static struct xtables_match socket_reg6 = { + .name = "socket", + .family = AF_INET6, + .version = IPTABLES_VERSION, + .size = XT_ALIGN(0), + .userspacesize = XT_ALIGN(0), + .parse = socket_parse, + .final_check = socket_check, + .print = socket_print, +}; + +void _init(void) +{ + xtables_register_match(&socket_reg); + xtables_register_match(&socket_reg6); +}
On Jul 31 2007 12:31, Laszlo Attila Toth wrote:
MD5 checksums of the tarballs:
a655fd090c06827bd6927f2b894ee178 tproxy-4.0.0-2.6.23-rc1.tgz d7175531d67beaab48293cd86cac5d21 tproxy-4.0.0-ubuntu-2.6.17-12.39.tgz
That can not compile, because there are still a lot of merge markers. @@ -1950,7 +2065,14 @@ int ip_route_input(struct sk_buff *skb, __be32 daddr, rth->fl.fl4_src == saddr && rth->fl.iif == iif && rth->fl.oif == 0 && +<<<<<<< HEAD:net/ipv4/route.c rth->fl.mark == skb->mark && +======= +#ifdef CONFIG_IP_ROUTE_FWMARK + rth->fl.fl4_fwmark == skb->nfmark && +#endif + !(rth->u.dst.flags & DST_DIVERTED) && +>>>>>>> 2bc8cb4... initial import of tproxy4 (2007-07-30):net/ipv4/route.c rth->fl.fl4_tos == tos) { rth->u.dst.lastuse = jiffies; dst_hold(&rth->u.dst); Jan --
On Tue, 2007-07-31 at 12:45 +0200, Jan Engelhardt wrote:
On Jul 31 2007 12:31, Laszlo Attila Toth wrote:
MD5 checksums of the tarballs:
a655fd090c06827bd6927f2b894ee178 tproxy-4.0.0-2.6.23-rc1.tgz d7175531d67beaab48293cd86cac5d21 tproxy-4.0.0-ubuntu-2.6.17-12.39.tgz
That can not compile, because there are still a lot of merge markers.
@@ -1950,7 +2065,14 @@ int ip_route_input(struct sk_buff *skb, __be32 daddr, rth->fl.fl4_src == saddr && rth->fl.iif == iif && rth->fl.oif == 0 && +<<<<<<< HEAD:net/ipv4/route.c rth->fl.mark == skb->mark && +======= +#ifdef CONFIG_IP_ROUTE_FWMARK + rth->fl.fl4_fwmark == skb->nfmark && +#endif + !(rth->u.dst.flags & DST_DIVERTED) && +>>>>>>> 2bc8cb4... initial import of tproxy4 (2007-07-30):net/ipv4/route.c rth->fl.fl4_tos == tos) { rth->u.dst.lastuse = jiffies; dst_hold(&rth->u.dst);
Looks like we did not resolve all conflicts when forward-porting to 2.6.23. The version on top of Ubuntu 2.6.17-12.39 was the one that has been tested, but we thought that we should release to a more current version as well. So in summary, the 2.6.17 based patch should be considered 'reasonably' stable, the other is completely untested. By the way, let me introduce Panther, he is going to be the new tproxy maintainer. As an additional item of interest, we've also published an experimental git tree to http://people.balabit.hu/panther/tproxy4.git/ -- Bazsi
On Jul 31 2007 13:13, Balazs Scheidler wrote:
Looks like we did not resolve all conflicts when forward-porting to 2.6.23.
The version on top of Ubuntu 2.6.17-12.39 was the one that has been tested, but we thought that we should release to a more current version as well.
So in summary, the 2.6.17 based patch should be considered 'reasonably' stable, the other is completely untested.
Is there a 'socket' match at all in balabit's tree? As far as I understand, I need xt_socket because otherwise, traffic to [foreign address on local socket] is forwarded to the real host.
By the way, let me introduce Panther, he is going to be the new tproxy maintainer.
As an additional item of interest, we've also published an experimental git tree to http://people.balabit.hu/panther/tproxy4.git/
403. Not a good day, today, is it? :) Jan --
On Tue, 2007-07-31 at 13:38 +0200, Jan Engelhardt wrote:
On Jul 31 2007 13:13, Balazs Scheidler wrote:
Looks like we did not resolve all conflicts when forward-porting to 2.6.23.
The version on top of Ubuntu 2.6.17-12.39 was the one that has been tested, but we thought that we should release to a more current version as well.
So in summary, the 2.6.17 based patch should be considered 'reasonably' stable, the other is completely untested.
Is there a 'socket' match at all in balabit's tree? As far as I understand, I need xt_socket because otherwise, traffic to [foreign address on local socket] is forwarded to the real host.
The socket match was one of the latest bits of Hidden's work, in an attempt to get tproxy merged upstream in a rush. However this makes tproxy more difficult to use. The exact change was to drop our routing changes and use connection mark to divert traffic from FORWARD to INPUT. This requires a couple of rules here and there, among others a rule with a 'socket' match. As we took over tproxy maintenance, we reverted back to the original scenario, using the routing changes as well. This means that you don't need 'socket' match right now. It does not mean that socket will never be reintroduced, I only want a functional, stable tproxy4 version first, and then talk to DaveM and Patrick about its possible inclusion in October, at Netfilter Developers' Workshop, whether they really insist not including our routing changes.
By the way, let me introduce Panther, he is going to be the new tproxy maintainer.
As an additional item of interest, we've also published an experimental git tree to http://people.balabit.hu/panther/tproxy4.git/
403. Not a good day, today, is it? :)
git clone works for me here, you don't need an index page in order to git clone to work. -- Bazsi
On Jul 31 2007 16:29, Balazs Scheidler wrote:
By the way, let me introduce Panther, he is going to be the new tproxy maintainer.
As an additional item of interest, we've also published an experimental git tree to http://people.balabit.hu/panther/tproxy4.git/
403. Not a good day, today, is it? :)
git clone works for me here, you don't need an index page in order to git clone to work.
Ah right. Well, if I see http: I usually point my webbrowser to it naturally. git clone is slow on http, maybe there's room for a git port? Jan --
On Tuesday 31 July 2007 12.45.15 Jan Engelhardt wrote:
On Jul 31 2007 12:31, Laszlo Attila Toth wrote:
MD5 checksums of the tarballs:
a655fd090c06827bd6927f2b894ee178 tproxy-4.0.0-2.6.23-rc1.tgz d7175531d67beaab48293cd86cac5d21 tproxy-4.0.0-ubuntu-2.6.17-12.39.tgz
That can not compile, because there are still a lot of merge markers.
Jan
Corrected in the git repository. Since the kernel isn't stable yet, it may have problems. I'm porting back the pach to 2.6.22. -- Regards, Laszlo Attila Toth
On Tue, Jul 31, 2007 at 12:31:35PM +0200, Laszlo Attila Toth wrote:
TProxy version 4.0.0 has been released for Linux 2.6.17 (Ubuntu 2.6.17.1-12.39) and 2.6.23-rc1
I downloaded the TProxy version 4.0.0 tarball for 2.6.17, patched and built myself a kernel. I am now trying to get a test foreign-tcp-connect program working to test my kernel. But, I must be missing something, because my test program is not working. Here is what I've done: - loaded the tproxy modules on the client test machine # modprobe xt_proxy # modprobe iptable_tproxy tproxy_any=1 - added a static arp entry for the foreign address on my web server pointing to the client test machine - execute my c program on the client machine, which essentially does the following (I can post the code if it is helpful) sock = socket(AF_INET, SOCK_STREAM, 0); setsockopt(sock, SOL_IP, IP_FREEBIND, &socktop, sizeof(sockopt)); /* local bind */ bind( sock, FOREIGN_IP, FOREIGN_PORT ); connect( sock, REMOTE_IP, REMOTE_PORT ); The connect hangs while the client machine (running the foreign connect program) sends out ARP packets asking who-has the foriegn address. What am I missing in the above steps to get my foreign-connect program working? Lastly, I noticed the set of diffs published does not include the IP_TRANSPARENT socket option. Has this setsockopt been dropped? Thanks, Cam
On Thursday 02 August 2007 00.03.35 Cameron Schaus wrote:
On Tue, Jul 31, 2007 at 12:31:35PM +0200, Laszlo Attila Toth wrote:
TProxy version 4.0.0 has been released for Linux 2.6.17 (Ubuntu 2.6.17.1-12.39) and 2.6.23-rc1
I downloaded the TProxy version 4.0.0 tarball for 2.6.17, patched and built myself a kernel. I am now trying to get a test foreign-tcp-connect program working to test my kernel. But, I must be missing something, because my test program is not working.
Here is what I've done:
- loaded the tproxy modules on the client test machine # modprobe xt_proxy # modprobe iptable_tproxy tproxy_any=1
- added a static arp entry for the foreign address on my web server pointing to the client test machine
- execute my c program on the client machine, which essentially does the following (I can post the code if it is helpful)
sock = socket(AF_INET, SOCK_STREAM, 0); setsockopt(sock, SOL_IP, IP_FREEBIND, &socktop, sizeof(sockopt)); /* local bind */ bind( sock, FOREIGN_IP, FOREIGN_PORT ); connect( sock, REMOTE_IP, REMOTE_PORT );
The connect hangs while the client machine (running the foreign connect program) sends out ARP packets asking who-has the foriegn address.
What am I missing in the above steps to get my foreign-connect program working?
Hello, For binding to foreign address you should echo 1 into /proc/sys/net/ipv4/ip_nonlocal_bind file, this may help. By the way if you set the IP_FREEBIND you don't need to use tproxy_any=1 module parameter. In the tproxy table it is tested whether either IP_FREEBIND or tproxy_any is set or not. Hm, it seems I should write these into the README file which is still a bit outdated.
Lastly, I noticed the set of diffs published does not include the IP_TRANSPARENT socket option. Has this setsockopt been dropped?
Yes, it is dropped. It is unnecessary since IP_FREEBIND is enough for transparent proxying.
Thanks, Cam
-- Regards, Laszlo Attila Toth
On Thu, Aug 02, 2007 at 01:25:39PM +0200, Laszlo Attila Toth wrote:
For binding to foreign address you should echo 1 into /proc/sys/net/ipv4/ip_nonlocal_bind file, this may help.
By the way if you set the IP_FREEBIND you don't need to use tproxy_any=1 module parameter. In the tproxy table it is tested whether either IP_FREEBIND or tproxy_any is set or not.
I have tried each of the above methods, but there are still problems. I have tracked the problem down the my use of a bridge. The foreign connect worked correctly on my test machines when I configured a single interface, eth1, with a local IP address, bound to the foreign address and issued the connect. When I configured a bridge, and repeated the test, I see the same behaviour I did yesterday. The machine trying to connect ends up arping the foriegn IP address. Here is what I'm doing: # modprobe xt_tproxy # modprobe iptable_tproxy # sysctl -w net.ipv4.ip_non_local_bind=1 # modprobe bridge # brctl addbr br0 # brctl addif br0 eth0 # brctl addif br0 eth1 # ifconfig br0 LOCAL_IP netmask LOCAL_MASK # add static arp for foreign address on web server # ./foreign-connect (this issues a bind to the foreign IP prior to connecting) Where can I look to correct this issue? Is iptables prerouting too late when dealing with packets arriving on a bridge interface? Thanks, Cam
On Aug 2 2007 11:55, Cameron Schaus wrote:
Where can I look to correct this issue? Is iptables prerouting too late when dealing with packets arriving on a bridge interface?
No, but in a bridge, packets get bridged rather than routed. Make sure they are getting routed. Jan --
Jan Engelhardt wrote:
No, but in a bridge, packets get bridged rather than routed. Make sure they are getting routed.
How can I make sure packets get routed instead of bridged? Does this involve using the brouting feature of ebtables? Thanks, Cam
On Aug 2 2007 12:07, Cameron Schaus wrote:
Jan Engelhardt wrote:
No, but in a bridge, packets get bridged rather than routed. Make sure they are getting routed.
How can I make sure packets get routed instead of bridged? Does this involve using the brouting feature of ebtables?
Yes. In some network layouts, you can use a half-bridge (which is a cool thing but probably requires the same number of steps) to reduce the number of needed ebt rules. Jan --
Jan Engelhardt wrote:
How can I make sure packets get routed instead of bridged? Does this involve using the brouting feature of ebtables?
Yes.
In some network layouts, you can use a half-bridge (which is a cool thing but probably requires the same number of steps) to reduce the number of needed ebt rules.
Thanks, using ebtables brouting does appear to work. However, it may be cumbersome for my application to use ebtables rules to keep certain traffic off the bridge while allowing unproxied traffic across the bridge. I have a proxy application running on the bridge IP address, and iptables rules currently direct traffic of interest to the proxy. I want to use the tproxy code to perform a foreign connect using the client's IP address. So the client's and server's addresses are more or less arbitrary. This makes generating ebtables brouting rules that preserve the bridge behavior for non-proxied traffic difficult. What is currently stopping the packets from being diverted locally once they enter the bridge? As I understand it now, the packets destined for the foreign IP address should enter the bridge, and since they are destined for the bridge MAC address they should enter the correct protocol handler. At some point the frames should enter IP tables PREROUTING where the tproxy code is invoked. However, something isn't right. Perhaps it's because the packet device is replaced with the bridge device and this doesn't match the socket information? I'd like to understand what isn't working in this case. One way around this is to add a check in br_handle_frame to get the socket and check if inet->freebind is set, similar to the tproxy prerouting code. One drawback of this is that fragmented IP packets cannot be reassembled to obtain the IP addresses and ports, and would end up lost in the stack. An ebtables module could likely be written to avoid modifying the bridge code. Are there better ways of getting these foreign packets into the network stack when they are destined for a local bridge interface? Thanks, Cam
On Aug 2 2007 15:45, Cameron Schaus wrote:
Jan Engelhardt wrote:
How can I make sure packets get routed instead of bridged? Does this involve using the brouting feature of ebtables?
Yes.
In some network layouts, you can use a half-bridge (which is a cool thing but probably requires the same number of steps) to reduce the number of needed ebt rules.
Thanks, using ebtables brouting does appear to work. However, it may be cumbersome for my application to use ebtables rules to keep certain traffic off the bridge while allowing unproxied traffic across the bridge.
I have a proxy application running on the bridge IP address, and iptables rules currently direct traffic of interest to the proxy. I want to use the tproxy code to perform a foreign connect using the client's IP address. So the client's and server's addresses are more or less arbitrary. This makes generating ebtables brouting rules that preserve the bridge behavior for non-proxied traffic difficult. [...]
I'll just give an example case: eth0 -- ISP-side -- my address 192.168.34.2 Objects: Exactly one ISP router at 192.168.34.1 eth1 -- LAN -- my address 192.168.34.2 Objects: Any number of clients in 192.168.34.0/24 And we'd like to filter some URLs with a squid running on .34.2. Step 1: Interfaces (of course you do this with your distro-specific config files), but the outline is: ip a a 192.168.34.1/32 dev eth0 ip a a 192.168.34.0/24 dev eth1 Yes, we have got overlapping networks, but that does not hurt, as the one with the tighter prefix length (32 wins over 24) has higher precedence. As I said, this cannot be used for all setups (especially where both segments contain changing hosts). So everything works as usual. On the clients, you may need an extra route: ip r a 192.168.34.1/32 via 192.168.34.2 but only if they should send packets there. [Default gateway for clients is of course .34.2.] Step 2: Configure eth0 to do ARP reply # Our so-called "half bridge" brctl addbr br0 brctl addif br0 eth0 # # ARP requests need to be handled by ebtables - hence they # _do_ need to be bridged (ACCEPT). # ebtables -t broute -P BROUTING DROP; ebtables -t broute -A BROUTING -p arp \ --opcode request -j ACCEPT; ebtables -t nat -A PREROUTING -p arp --opcode request \ -j arpreply --arpreply-mac `cat /sys/class/net/br0/address` \ --arpreply-target DROP; Note that you can also bridge all ARP traffic, i.e. let the clients answer ARP. I do not consider that necessary. That's it already. On the .34.2 server, squid is run with server-side-transparency ("tproxy") and, optionally client-side-transparency if wanted. The benefit of this is that no packet can ever be accidentally bridged ("slip through"), because the bridge consists of only one port. The other benefit (of using DROP in broute) is that packets come through regular interfaces instead of br0, making accounting more precise, and you do not have to use -m physdev in iptables.
What is currently stopping the packets from being diverted locally once they enter the bridge? As I understand it now, the packets destined for the foreign IP address should enter the bridge, and since they are destined for the bridge MAC address
Generally, people set it up differently, so that br0 encompasses both eth0 and eth1. It works too, but needs a bit more thought: Your bridge sends a packet with a foreign address. So the ISP router will ask "arp who-has .34.99", and a client responds with ".34.99 at MAC macof3499". Then the router will send whatever IPv4 packet it wanted to send -- to macof3499, NOT to macofbridge. Hence, the packet gets bridged through the blue level (see http://www.imagestream.com/~josh/PacketFlow-new.png ). But for tproxying to be effective, the packet needs to hit the "routing decision (PRDB)" circle.[1] There are ways to achieve that: - DROP non ARP-packets in the broute table (= inject them into the route stack - and hence tproxy) path: bridging process, brouting, routing process, Green Level. - ebt_dnat packets in the nat table (= change macof3499 to macofbridge) path: bridging process, brouting, blue prerouting, bridging decision, *then* moves up to the Green Level Hope this explains things a bit :-)
they should enter the correct protocol handler. At some point the frames should enter IP tables PREROUTING where the tproxy code is invoked. However, something isn't right. Perhaps it's because the packet device is replaced with the bridge device and this doesn't match the socket information? I'd like to understand what isn't working in this case.
One way around this is to add a check in br_handle_frame to get the socket and check if inet->freebind is set, similar to the tproxy prerouting code. One drawback of this is that fragmented IP packets cannot be reassembled to obtain the IP addresses and ports, and would end up lost in the stack. An ebtables module could likely be written to avoid modifying the bridge code.
Are there better ways of getting these foreign packets into the network stack when they are destined for a local bridge interface?
Thanks, Cam
[ And if you are concerned about xt_socket from Krisztian's tproxy, well, that's related to the routing process, where we also have to either send the packet upwards (into INPUT - yes we want this) - since if we did not, it would be forwarded to the right to the real client. ] Jan --
Thanks for the detailed reply. Jan Engelhardt wrote:
Your bridge sends a packet with a foreign address. So the ISP router will ask "arp who-has .34.99", and a client responds with ".34.99 at MAC macof3499". Then the router will send whatever IPv4 packet it wanted to send -- to macof3499, NOT to macofbridge.
I agree with the above statements. However, for my testing, I added a static arp entry of the foreign IP address (.34.99, from your example) with the MAC address of my bridge. So the packet should arrive at the bridge with the dest MAC set to the bridge's MAC address. It sounds like this should be enough to cause the packet to be sent back up the stack to my test code. But what I see is that my bridge ends up sending out arp requests for the foreign IP address (.34.9). So this is not the expected behaviour?
Hope this explains things a bit :-) Yes, your reply was very helpful. I'm off for a few days, but I'll pick this stuff back up again when I get back.
Thanks again for the information. Cam
On Aug 2 2007 16:56, Cameron Schaus wrote:
Jan Engelhardt wrote:
Your bridge sends a packet with a foreign address. So the ISP router will ask "arp who-has .34.99", and a client responds with ".34.99 at MAC macof3499". Then the router will send whatever IPv4 packet it wanted to send -- to macof3499, NOT to macofbridge.
I agree with the above statements. However, for my testing, I added a static arp entry
Well, _where_ did you add this entry? It must be added to .34.1, not .34.2 and not .34.99. And that raises problems, for example: - .34.1 is not under your control and/or has no way to add ARP entries (like most routers which only have a web interface) - there is more than just .34.1, for example if br0[eth0+eth1] were the combined .34.0/24 (no router so to speak), then you would have to add ARP for all machines on all machines (yikes!)
of the foreign IP address (.34.99, from your example) with the MAC address of my bridge. So the packet should arrive at the bridge with the dest MAC set to the bridge's MAC address.
It sounds like this should be enough to cause the packet to be sent back up the stack to my test code. But what I see is that my bridge ends up sending out arp requests for the foreign IP address (.34.9). So this is not the expected behaviour?
Jan --
Hi all, sorry, for me english. Following, i am with problem in the hour to compile kernel last version 2.6.23 more patch tproxy-4 last version too, without patch the kernel compile very well, but application with the patch don't compile, generating error low: proxyserver:/usr/src/linux# make scripts/kconfig/conf -s arch/i386/Kconfig CHK include/linux/version.h CHK include/linux/utsrelease.h CALL scripts/checksyscalls.sh CC init/main.o In file included from include/linux/netlink.h:139, from include/linux/genetlink.h:4, from include/net/genetlink.h:4, from include/linux/taskstats_kern.h:12, from init/main.c:45: include/linux/skbuff.h:292: error: expected specifier-qualifier-list before 'ip_tproxy' include/linux/skbuff.h: In function 'skb_truesize_check': include/linux/skbuff.h:390: error: 'struct sk_buff' has no member named 'truesize' include/linux/skbuff.h: In function 'skb_end_pointer': include/linux/skbuff.h:429: error: 'const struct sk_buff' has no member named 'end' include/linux/skbuff.h: In function 'skb_get': include/linux/skbuff.h:456: error: 'struct sk_buff' has no member named 'users' include/linux/skbuff.h: In function 'skb_shared': include/linux/skbuff.h:522: error: 'const struct sk_buff' has no member named 'users' include/linux/skbuff.h: In function 'skb_tail_pointer': include/linux/skbuff.h:872: error: 'const struct sk_buff' has no member named 'tail' include/linux/skbuff.h: In function 'skb_reset_tail_pointer': include/linux/skbuff.h:877: error: 'struct sk_buff' has no member named 'tail' include/linux/skbuff.h:877: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'skb_set_tail_pointer': include/linux/skbuff.h:882: error: 'struct sk_buff' has no member named 'tail' include/linux/skbuff.h:882: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function '__skb_put': include/linux/skbuff.h:894: error: 'struct sk_buff' has no member named 'tail' include/linux/skbuff.h: In function 'skb_put': include/linux/skbuff.h:912: error: 'struct sk_buff' has no member named 'tail' include/linux/skbuff.h:914: error: 'struct sk_buff' has no member named 'tail' include/linux/skbuff.h:914: error: 'struct sk_buff' has no member named 'end' include/linux/skbuff.h: In function '__skb_push': include/linux/skbuff.h:921: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h:923: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'skb_push': include/linux/skbuff.h:937: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h:939: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h:939: error: 'struct sk_buff' has no member named 'head' include/linux/skbuff.h:941: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function '__skb_pull': include/linux/skbuff.h:948: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function '__pskb_pull': include/linux/skbuff.h:974: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'skb_headroom': include/linux/skbuff.h:999: error: 'const struct sk_buff' has no member named 'data' include/linux/skbuff.h:999: error: 'const struct sk_buff' has no member named 'head' include/linux/skbuff.h: In function 'skb_tailroom': include/linux/skbuff.h:1010: error: 'const struct sk_buff' has no member named 'end' include/linux/skbuff.h:1010: error: 'const struct sk_buff' has no member named 'tail' include/linux/skbuff.h: In function 'skb_reserve': include/linux/skbuff.h:1023: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h:1024: error: 'struct sk_buff' has no member named 'tail' include/linux/skbuff.h: In function 'skb_transport_header': include/linux/skbuff.h:1086: error: 'const struct sk_buff' has no member named 'transport_header' include/linux/skbuff.h: In function 'skb_reset_transport_header': include/linux/skbuff.h:1091: error: 'struct sk_buff' has no member named 'transport_header' include/linux/skbuff.h:1091: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'skb_set_transport_header': include/linux/skbuff.h:1097: error: 'struct sk_buff' has no member named 'transport_header' include/linux/skbuff.h:1097: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'skb_network_header': include/linux/skbuff.h:1102: error: 'const struct sk_buff' has no member named 'network_header' include/linux/skbuff.h: In function 'skb_reset_network_header': include/linux/skbuff.h:1107: error: 'struct sk_buff' has no member named 'network_header' include/linux/skbuff.h:1107: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'skb_set_network_header': include/linux/skbuff.h:1112: error: 'struct sk_buff' has no member named 'network_header' include/linux/skbuff.h:1112: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'skb_mac_header': include/linux/skbuff.h:1117: error: 'const struct sk_buff' has no member named 'mac_header' include/linux/skbuff.h: In function 'skb_mac_header_was_set': include/linux/skbuff.h:1122: error: 'const struct sk_buff' has no member named 'mac_header' include/linux/skbuff.h: In function 'skb_reset_mac_header': include/linux/skbuff.h:1127: error: 'struct sk_buff' has no member named 'mac_header' include/linux/skbuff.h:1127: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'skb_set_mac_header': include/linux/skbuff.h:1132: error: 'struct sk_buff' has no member named 'mac_header' include/linux/skbuff.h:1132: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'skb_transport_offset': include/linux/skbuff.h:1138: error: 'const struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'skb_network_header_len': include/linux/skbuff.h:1143: error: 'const struct sk_buff' has no member named 'transport_header' include/linux/skbuff.h:1143: error: 'const struct sk_buff' has no member named 'network_header' include/linux/skbuff.h: In function 'skb_network_offset': include/linux/skbuff.h:1148: error: 'const struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'skb_orphan': include/linux/skbuff.h:1260: error: 'struct sk_buff' has no member named 'destructor' include/linux/skbuff.h:1261: error: 'struct sk_buff' has no member named 'destructor' include/linux/skbuff.h:1262: error: 'struct sk_buff' has no member named 'destructor' include/linux/skbuff.h: In function 'skb_header_pointer': include/linux/skbuff.h:1551: error: 'const struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'skb_copy_from_linear_data': include/linux/skbuff.h:1563: error: 'const struct sk_buff' has no member named 'data' include/linux/skbuff.h:1563: error: 'const struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'skb_copy_from_linear_data_offset': include/linux/skbuff.h:1570: error: 'const struct sk_buff' has no member named 'data' include/linux/skbuff.h:1570: error: 'const struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'skb_copy_to_linear_data': include/linux/skbuff.h:1577: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h:1577: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'skb_copy_to_linear_data_offset': include/linux/skbuff.h:1585: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h:1585: error: 'struct sk_buff' has no member named 'data' include/linux/skbuff.h: In function 'nf_conntrack_get_reasm': include/linux/skbuff.h:1664: error: 'struct sk_buff' has no member named 'users' include/linux/skbuff.h: In function 'nf_reset': include/linux/skbuff.h:1687: error: 'struct sk_buff' has no member named 'nfct' include/linux/skbuff.h:1688: error: 'struct sk_buff' has no member named 'nfct' include/linux/skbuff.h:1689: error: 'struct sk_buff' has no member named 'nfct_reasm' include/linux/skbuff.h:1690: error: 'struct sk_buff' has no member named 'nfct_reasm' include/linux/skbuff.h: In function '__nf_copy': include/linux/skbuff.h:1702: error: 'struct sk_buff' has no member named 'nfct' include/linux/skbuff.h:1702: error: 'const struct sk_buff' has no member named 'nfct' include/linux/skbuff.h:1703: error: 'const struct sk_buff' has no member named 'nfct' include/linux/skbuff.h:1705: error: 'struct sk_buff' has no member named 'nfct_reasm' include/linux/skbuff.h:1705: error: 'const struct sk_buff' has no member named 'nfct_reasm' include/linux/skbuff.h:1706: error: 'const struct sk_buff' has no member named 'nfct_reasm' include/linux/skbuff.h: In function 'nf_copy': include/linux/skbuff.h:1717: error: 'struct sk_buff' has no member named 'nfct' include/linux/skbuff.h:1718: error: 'struct sk_buff' has no member named 'nfct_reasm' In file included from include/linux/genetlink.h:4, from include/net/genetlink.h:4, from include/linux/taskstats_kern.h:12, from init/main.c:45: include/linux/netlink.h: In function 'nlmsg_hdr': include/linux/netlink.h:143: error: 'const struct sk_buff' has no member named 'data' In file included from include/net/genetlink.h:5, from include/linux/taskstats_kern.h:12, from init/main.c:45: include/net/netlink.h: In function 'nlmsg_trim': include/net/netlink.h:560: error: 'struct sk_buff' has no member named 'data' make[1]: ** [init/main.o] Erro 1 make: ** [init] Erro 2
Hello, Sorry for that. The git repository contains a newer version of the tproxy patch but at least one bug remains. I ported it to vanilla kernel 2.6.22. Monday I will push the code to the repository after I finish and test it. There is no siginificant difference between 2.6.22 and 2.6.23-rc1 but the stable kernel is preferred. The git repository is at: http://people.balabit.hu/panther/tproxy4.git/ cpd@agitajau.com.br írta:
Hi all,
sorry, for me english.
Following, i am with problem in the hour to compile kernel last version 2.6.23 more patch tproxy-4 last version too, without patch the kernel compile very well, but application with the patch don't compile, generating error low:
proxyserver:/usr/src/linux# make scripts/kconfig/conf -s arch/i386/Kconfig CHK include/linux/version.h CHK include/linux/utsrelease.h CALL scripts/checksyscalls.sh CC init/main.o In file included from include/linux/netlink.h:139, from include/linux/genetlink.h:4, from include/net/genetlink.h:4, from include/linux/taskstats_kern.h:12, from init/main.c:45: include/linux/skbuff.h:292: error: expected specifier-qualifier-list before 'ip_tproxy' include/linux/skbuff.h: In function 'skb_truesize_check': include/linux/skbuff.h:390: error: 'struct sk_buff' has no member named 'truesize'
-- Regards, Laszlo Attila Toth
Jan Engelhardt wrote:
Well, _where_ did you add this entry? It must be added to .34.1, not .34.2 and not .34.99. And that raises problems, for example:
- .34.1 is not under your control and/or has no way to add ARP entries (like most routers which only have a web interface)
- there is more than just .34.1, for example if br0[eth0+eth1] were the combined .34.0/24 (no router so to speak), then you would have to add ARP for all machines on all machines (yikes!)
I add the static arp entry on my web server. I am doing this purely for testing purposes. In the real system, the arp traffic flows across the bridge so the webserver has the client's MAC address. To be clear, I have one test bridge running the tproxy code generating an HTTP GET request to a web server using a foreign source address for the request. I have placed the static arp entry on the web server. Using my setup I have validated that the tproxy v4.0.0 code does work without a bridge. However, when I configure a bridge, per my previous posts, and run my test again, the client machine (with bridge+tproxy) sits and sends arp requests for the foreign IP address. I would like to understand why the 4.0.0 code does not work when a bridge is involved, because based on previous discussions it sounds like should work when I create a static ARP entry on the web server. The packets arriving back at the bridge have a dest ethernet address of the bridge, and so should make it up the stack correctly. When I use ebtables brouting feature, the packets are processed correctly by the bridge, however, using ebtables brouting is not feasible for my application. Thanks, Cam
Hi, On k, aug 07, 2007 at 09:55:29 -0600, Cameron Schaus wrote:
To be clear, I have one test bridge running the tproxy code generating an HTTP GET request to a web server using a foreign source address for the request. I have placed the static arp entry on the web server.
Using my setup I have validated that the tproxy v4.0.0 code does work without a bridge. However, when I configure a bridge, per my previous posts, and run my test again, the client machine (with bridge+tproxy) sits and sends arp requests for the foreign IP address.
I would like to understand why the 4.0.0 code does not work when a bridge is involved, because based on previous discussions it sounds like should work when I create a static ARP entry on the web server. The packets arriving back at the bridge have a dest ethernet address of the bridge, and so should make it up the stack correctly.
When I use ebtables brouting feature, the packets are processed correctly by the bridge, however, using ebtables brouting is not feasible for my application.
If you configure ARP properly, then it should work. Definitely worth investigating... Do you have any clue _where_ things might go wrong inside the kernel? (It looks like the packet does get up to routing, but then the socket is somehow not found and the kernel tries to forward it.) -- KOVACS Krisztian
KOVACS Krisztian wrote:
If you configure ARP properly, then it should work. Definitely worth investigating... Do you have any clue _where_ things might go wrong inside the kernel? (It looks like the packet does get up to routing, but then the socket is somehow not found and the kernel tries to forward it. I speculate that the problem lies with the interface checking during socket lookup. I speculate that when the outgoing socket is created, the interface is the bridge interface, br0. When the packets arrive through the bridge, the bridge may set the interface the the physical interface the packet arrived on, eth0 or eth1. Hence the prerouting socket lookup won't find the socket. This is a guess right now. I will attempt to verify this is the case soon.
Cam
KOVACS Krisztian wrote:
I would like to understand why the 4.0.0 code does not work when a bridge is involved, because based on previous discussions it sounds like should work when I create a static ARP entry on the web server. The packets arriving back at the bridge have a dest ethernet address of the bridge, and so should make it up the stack correctly.
When I use ebtables brouting feature, the packets are processed correctly by the bridge, however, using ebtables brouting is not feasible for my application.
If you configure ARP properly, then it should work. Definitely worth investigating... Do you have any clue _where_ things might go wrong inside the kernel? (It looks like the packet does get up to routing, but then the socket is somehow not found and the kernel tries to forward it.)
I added some debug information to the ip_tproxy_prerouting function, and it appears that the incoming connection is found in the socket hash, and that "freebind" is set in the struct. So, my initial speculation about mismatched devices was not correct. Do you have anymore ideas about why this code does not work with a bridge? I'm going to investigate the ip_divert_local function, but I'm not 100% clear about what it's doing to "short circut" the routing. Thanks, Cam
On Mon, 2007-08-13 at 16:49 -0600, Cameron Schaus wrote:
KOVACS Krisztian wrote:
I would like to understand why the 4.0.0 code does not work when a bridge is involved, because based on previous discussions it sounds like should work when I create a static ARP entry on the web server. The packets arriving back at the bridge have a dest ethernet address of the bridge, and so should make it up the stack correctly.
When I use ebtables brouting feature, the packets are processed correctly by the bridge, however, using ebtables brouting is not feasible for my application.
If you configure ARP properly, then it should work. Definitely worth investigating... Do you have any clue _where_ things might go wrong inside the kernel? (It looks like the packet does get up to routing, but then the socket is somehow not found and the kernel tries to forward it.)
I added some debug information to the ip_tproxy_prerouting function, and it appears that the incoming connection is found in the socket hash, and that "freebind" is set in the struct. So, my initial speculation about mismatched devices was not correct.
Do you have anymore ideas about why this code does not work with a bridge?
I'm going to investigate the ip_divert_local function, but I'm not 100% clear about what it's doing to "short circut" the routing.
It sets up a dst_entry that forces the packet go to ip_local_deliver(), which is the main entry point of the IP stack in the kernel. I don't see how this could go wrong from that point. The prerouting hook of tproxy looks up the socket and stores this information in skb->sk, which in turn will be used by the TCP stack instead of doing another lookup. Can you confirm that skb->sk is set in tcp_v4_rcv() by the time the packet gets there? You could also get some information by adding a "LOG" rule in filter/INPUT, as that chain is iterated only if the packet successfully got to the input side of the IP stack. -- Bazsi
Balazs Scheidler wrote:
Can you confirm that skb->sk is set in tcp_v4_rcv() by the time the packet gets there?
When a bridge is used, the packets never arrive in tcp_v4_rcv. The packets make it through the ip_divert_local function, initially by creating the dst entry (the bottom logic of the function), and subsequently by looking up the cached entry. But they never arrive at tcp_v4_rcv. When I "short circut" the bridge by using an ebtables entry, I see the packets going through tcp_v4_rcv. Is there a mismatch with the bridge device and the "in" device used in pre-routing that prevents the packet from being dispatched to the ip protocol handler? Any other ideas?
You could also get some information by adding a "LOG" rule in filter/INPUT, as that chain is iterated only if the packet successfully got to the input side of the IP stack.
No packets are logged when I add the LOG rule as you suggest, likely because they are not making it into the stack (tcp_v4_rcv). Any other ideas on where to look are appreciated. I'll try and trace the packet through the bridge code, but the bridge code is fairly simple, and doesn't do much to the packet before it re-injects it for local delivery. Cam
participants (7)
-
Balazs Scheidler
-
Cameron Schaus
-
cpd＠agitajau.com.br
-
Jan Engelhardt
-
KOVACS Krisztian
-
Laszlo Attila Toth
-
Tóth László Attila