Hello! We're using TPROXY 2.0.0 for 2.4.27 and are running into a strange issue which seems related to NAT reservation. As usual, we have a client TCP connection coming in from a given <saddr,sport> going to a particular <daddr,dport>. Netfilter rules REDIRECT this to our proxy code, which then uses TPROXY to connect out to <daddr,dport> using <saddr,sport> as the apparent source. We use the same operation sequence that appears in the test directory: first bind the socket to a local address, then do a TPROXY ASSIGN assigning <saddr,sport> to the socket, then TPROXY FLAGS to set ITP_CONNECT|ITP_ONCE, and then the actual connect(). With earlier versions of TPROXY (though I can't guarantee that kernel options haven't changed) this worked fine. Now, though, we see two problems. First, on when we make the call to ASSIGN, we get an error: IP_TPROXY: ip_tproxy_nat_reserve proto 6 foreign 10.0.0.2:32772 peer 0.0.0.0:0 IP_TPROXY: IP_TPROXY_ASSIGN cannot register NAT reservation 0200000a:0480 Apparently the NAT reservation is failing because when the initial TCP connection came in, conntrack set up a record expecting a reply to <saddr,sport>. The only way I could see to get around this was to set SO_REUSEADDR on the socket before the call to ASSIGN. However the second problem is that, if I do this, I get an error in the FLAGS call: IP_TPROXY: ip_tproxy_nat_reserve proto 6 foreign 10.0.0.2:32775 peer 0.0.0.0:0 IP_TPROXY: IP_TPROXY_FLAGS sr c660bf9c: failed to register NAT reservation I was able to work around these problems by first setting SO_REUSEADDR on the socket before the ASSIGN than clearing it before the FLAGS call, but that doesn't seem ideal. Is this how it's supposed to be done, or do I just have some basic misunderstanding about how TPROXY should be used? I have to admit I'm not clear on the purpose of NAT reservations. Thanks! Tim __________________________________ Do you Yahoo!? Check out the new Yahoo! Front Page. www.yahoo.com
Hi, 2004-11-08, h keltezéssel 03:28-kor Tim Burress ezt írta:
We're using TPROXY 2.0.0 for 2.4.27 and are running into a strange issue which seems related to NAT reservation. As usual, we have a client TCP connection coming in from a given <saddr,sport> going to a particular <daddr,dport>. Netfilter rules REDIRECT this to our proxy code, which then uses TPROXY to connect out to <daddr,dport> using <saddr,sport> as the apparent source.
We use the same operation sequence that appears in the test directory: first bind the socket to a local address, then do a TPROXY ASSIGN assigning <saddr,sport> to the socket, then TPROXY FLAGS to set ITP_CONNECT|ITP_ONCE, and then the actual connect().
With earlier versions of TPROXY (though I can't guarantee that kernel options haven't changed) this worked fine. Now, though, we see two problems. First, on when we make the call to ASSIGN, we get an error:
IP_TPROXY: ip_tproxy_nat_reserve proto 6 foreign 10.0.0.2:32772 peer 0.0.0.0:0 IP_TPROXY: IP_TPROXY_ASSIGN cannot register NAT reservation 0200000a:0480
Apparently the NAT reservation is failing because when the initial TCP connection came in, conntrack set up a record expecting a reply to <saddr,sport>. The only way I could see to get around this was to set SO_REUSEADDR on the socket before the call to ASSIGN.
However the second problem is that, if I do this, I get an error in the FLAGS call:
IP_TPROXY: ip_tproxy_nat_reserve proto 6 foreign 10.0.0.2:32775 peer 0.0.0.0:0 IP_TPROXY: IP_TPROXY_FLAGS sr c660bf9c: failed to register NAT reservation
I was able to work around these problems by first setting SO_REUSEADDR on the socket before the ASSIGN than clearing it before the FLAGS call, but that doesn't seem ideal. Is this how it's supposed to be done, or do I just have some basic misunderstanding about how TPROXY should be used? I have to admit I'm not clear on the purpose of NAT reservations.
Before using TPROXY_FLAGS you should specify the other endpoint of the new connection using TPROXY_CONNECT. Try something like this (augmented foreign-tcp-connect.c, completely untested): 54 /* assign foreign address */ 55 itp.op = TPROXY_ASSIGN; 56 inet_aton(FOREIGN_IP, (struct in_addr *) &itp.v.addr.faddr); 57 itp.v.addr.fport = htons(2000); 58 59 if (setsockopt(sock, SOL_IP, IP_TPROXY, &itp, sizeof(itp)) == -1) 60 { 61 perror("setsockopt(SOL_IP, IP_TPROXY, TPROXY_ASSIGN)"); 62 return -1; 63 } 64 65 /* specify peer endpoint: the same address we will pass to connect() */ 66 itp.op = TPROXY_CONNECT; 67 inet_aton(DEST_IP, (struct in_addr *) &itp.v.addr.faddr); 68 itp.v.addr.fport = htons(80); 69 if (setsockopt(sock, SOL_IP, IP_TPROXY, &itp, sizeof(itp)) == -1) 70 { 71 perror("setsockopt(SOL_IP, IP_TPROXY, TPROXY_CONNECT)"); 72 return -1; 73 } 74 75 /* set connect flag on socket */ 76 itp.op = TPROXY_FLAGS; 77 itp.v.flags = ITP_CONNECT | ITP_ONCE; 78 if (setsockopt(sock, SOL_IP, IP_TPROXY, &itp, sizeof(itp)) == -1) 79 { 80 perror("setsockopt(SOL_IP, IP_TPROXY, TPROXY_FLAGS)"); 81 return -1; 82 } 83 84 /* connect */ 85 inet_aton(DEST_IP, &sin.sin_addr); 86 sin.sin_port = htons(80); 87 if (connect(sock, (struct sockaddr *) &sin, sizeof(sin)) == -1) 88 { 89 perror("connect"); 90 return -1; 91 } -- Regards, Krisztian KOVACS
Hello! --- KOVACS Krisztian <hidden@balabit.hu> wrote:
Before using TPROXY_FLAGS you should specify the other endpoint of the new connection using TPROXY_CONNECT.
This works up to a point, but we run into trouble if the destination address is subject to a DNAT rule. In that case, if we use TPROXY_CONNECT to specify the remote endpoint, we kind of shoot ourselves in the foot because by the time TPROXY sees the packet in POSTROUTING, the packet's destination address will have changed. The workaround of playing games with SO_REUSEADDR seems to do OK in this situation, but it's ugly and I'm not sure what the side effects might be. Tim __________________________________ Do you Yahoo!? Check out the new Yahoo! Front Page. www.yahoo.com
participants (2)
-
KOVACS Krisztian
-
Tim Burress