- tproxy - lists.balabit.hu

Kernel compilation error
by Tomasz Wrona 09 Sep '03

09 Sep '03

Hello, I upgraded 2.4.21 kernel with latest tproxy patch but compilation process returns follownig error: ip_nat_h323.c: In function `h225_nat_expected': ip_nat_h323.c:135: too few arguments to function `ip_nat_setup_info_R4b726ec1' make[2]: *** [ip_nat_h323.o] Error 1 make[2]: Leaving directory `/home/src/linux-2.4.21/net/ipv4/netfilter' make[1]: *** [_modsubdir_ipv4/netfilter] Error 2 make[1]: Leaving directory `/home/src/linux-2.4.21/net' make: *** [_mod_net] Error 2 In configure I removed from options to compile h323 module but error appeared in other module, also remover but error still appears... What it can be ? Regards, tw --

2 1

Question about TCP RST and tproxy
by Stas Grabois 08 Sep '03

08 Sep '03

We have a problem with the following scenario: 1) int fd=socket(AF_INET, SOCK_STREAM, 0) make fd non-blocking bind fd to a local address an port 0 setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, 1); 2) setsockopt(fd, SOL_IP, IP_TPROXY_ASSIGN, {client_ip, client_port}); setsockopt(fd, SOL_IP, IP_TPROXY_FLAGS, ITP_CONNECT | ITP_ONCE); 3) connect(fd, {backend_ip, backend_port}); connect() return -1, errno=EAGAIN; 4) send(fd, buffer, sizeof(buffer), 0); send returns -1, errno=ECONNREFUSED; 5) setsockopt(fd, SOL_IP, IP_TPROXY_UNASSIGN); it returns -1, errno=EINVAL; Our error is the fact that we call send() without checking that connect() has failed - it fails because there is no server and we get RST. When Linux kernel receives RST - it releases socket's source port, and sets sk->num=0; After that send, which calls sendmsg in af_inet.c, chooses new source port via inet_autobind(). inet_autobind() selects new source port, which is used as hash key to unassign original socket, and the setsockopt(IP_TPROXY_UNASSIGN) fails to find the socket. While we will fix out problem with send()-after-connect(), I think that the same problem can happen on connected socket - when we call send() twice, and get RST between the first and the second send(). Only in this case the second send() will return ECONNRESET, but the source port will be changed anyway. How difficult is to add handling of RST packets to tproxy module? Some way to remove entries from the tproxy's hash tables on RST...

1 0

Question about TCP RST and tproxy
by Stas Grabois 08 Sep '03

08 Sep '03

We have a problem with the following scenario: 1) int fd=socket(AF_INET, SOCK_STREAM, 0) make fd non-blocking bind fd to a local address an port 0 setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, 1); 2) setsockopt(fd, SOL_IP, IP_TPROXY_ASSIGN, {client_ip, client_port}); setsockopt(fd, SOL_IP, IP_TPROXY_FLAGS, ITP_CONNECT | ITP_ONCE); 3) connect(fd, {backend_ip, backend_port}); connect() return -1, errno=EAGAIN; 4) send(fd, buffer, sizeof(buffer), 0); send returns -1, errno=ECONNREFUSED; 5) setsockopt(fd, SOL_IP, IP_TPROXY_UNASSIGN); it returns -1, errno=EINVAL; Our error is the fact that we call send() without checking that connect() has failed - it fails because there is no server and we get RST. When Linux kernel receives RST - it releases socket's source port, and sets sk->num=0; After that send, which calls sendmsg in af_inet.c, chooses new source port via inet_autobind(). inet_autobind() selects new source port, which is used as hash key to unassign original socket, and the setsockopt(IP_TPROXY_UNASSIGN) fails to find the socket. While we will fix out problem with send()-after-connect(), I think that the same problem can happen on connected socket - when we call send() twice, and get RST between the first and the second send(). Only in this case the second send() will return ECONNRESET, but the source port will be changed anyway. How difficult is to add handling of RST packets to tproxy module? Some way to remove entries from the tproxy's hash tables on RST...

1 0

RE: [tproxy] Question about warning message
by Yuval Pemper 27 Aug '03

27 Aug '03

Further to the email below, we've found an interim solution that fits our requirements: 1. We don't want any unNATed tproxy packets to be sent by our application. 2. We want NATed tproxy packets to be sent out ASAP. Our solution is to drop packets for which the call to ip_tproxy_setup_nat returns NF_DROP. In addition, we reduce the connection tracking time-wait timeout to 1 second and the initial RTO for SYN packets to 1.05 seconds (we took a spare of 50 msec). The simple patch for accomplishing the above (not including changing the connection tracking time-wait timeout) is given below. Obviously, a better solution would be to allow setting up tproxy NAT for conflicting time-wait connections, as described in my previous email. -- Yuval diff -Nru linux.org/include/net/tcp.h linux/include/net/tcp.h --- linux.org/include/net/tcp.h 2003-08-11 17:53:55.000000000 +0300 +++ linux/include/net/tcp.h 2003-08-27 09:18:22.000000000 +0300 @@ -334,6 +334,7 @@ #define TCP_RTO_MAX ((unsigned)(120*HZ)) #define TCP_RTO_MIN ((unsigned)(HZ/5)) #define TCP_TIMEOUT_INIT ((unsigned)(3*HZ)) /* RFC 1122 initial RTO value */ +#define TCP_SYN_TIMEOUT_INIT ((unsigned)(1.05*HZ)) /* SYN RETRANSMIT TIMEOUT */ #define TCP_RESOURCE_PROBE_INTERVAL ((unsigned)(HZ/2U)) /* Maximal interval between probes * for local resources. diff -Nru linux.org/net/ipv4/netfilter/iptable_tproxy.c linux/net/ipv4/netfilter/iptable_tproxy.c --- linux.org/net/ipv4/netfilter/iptable_tproxy.c 2003-08-11 17:53:24.000000000 +0300 +++ linux/net/ipv4/netfilter/iptable_tproxy.c 2003-08-27 09:16:40.000000000 +0300 @@ -581,7 +581,7 @@ * necessary NAT function */ - ip_tproxy_setup_nat(*pskb, hooknum, sr, !(sr->flags & TF_UNIDIR)); + verdict = ip_tproxy_setup_nat(*pskb, hooknum, sr, !(sr->flags & TF_UNIDIR)); if (sr->flags & TF_NAT_ONCE) { /* we were to be applied only once */ @@ -625,7 +625,7 @@ sr.lport = lport; sr.proto = iph->protocol; DEBUGP(KERN_DEBUG "IP_TPROXY: performing redirect to %08x:%04x\n", sr.laddr, sr.lport); - ip_tproxy_setup_nat(*pskb, hooknum, &sr, 0); + verdict = ip_tproxy_setup_nat(*pskb, hooknum, &sr, 0); } } } diff -Nru linux.org/net/ipv4/tcp_output.c linux/net/ipv4/tcp_output.c --- linux.org/net/ipv4/tcp_output.c 2003-01-31 14:13:49.000000000 +0200 +++ linux/net/ipv4/tcp_output.c 2003-08-27 09:17:02.000000000 +0300 @@ -1201,7 +1201,7 @@ tp->rcv_wup = 0; tp->copied_seq = 0; - tp->rto = TCP_TIMEOUT_INIT; + tp->rto = TCP_SYN_TIMEOUT_INIT; tp->retransmits = 0; tcp_clear_retrans(tp); } -- Yuval > -----Original Message----- > From: Yuval Pemper > Sent: Tue, August 26, 2003 8:43 > To: 'Balazs Scheidler' > Cc: tproxy(a)lists.balabit.hu; Sagi Grabois > Subject: RE: [tproxy] Question about warning message > > > Thanks for your reply. Several remarks: > > 1. We've seen the phenomenon of packets being sent unNATed. > We've also seen that the timeout period affecting this > phenomenon is the ip_ct_tcp_timeout_time_wait, which is set > to 120 seconds by default. We have changed this value to 1 > second to help us deal with this issue. > > 2. The fact that the connection tracking time-wait timeout > affects the amount of time in which unNATed packets are sent > out indicates that the connection was in the time-wait state > in the connection tracking table. We've seen this is also > true for connections whose TCP state was not time-wait, i.e., > connections that were passively closed by us. It seems that > the connection tracking and TCP state machine differ in this > point. Not sure whether this is intended or not. > > 3. Regarding more liberal source port allocation, RFC 1122 > (Requirements for Internet Hosts -- Communication Layers), > section 4.2.2.13, states: > > "When a connection is closed actively, it MUST linger in > TIME-WAIT state for a time 2xMSL (Maximum Segment Lifetime). > However, it MAY accept a new SYN from the remote TCP to > reopen the connection directly from TIME-WAIT state, if it: > > (1) assigns its initial sequence number for the new > connection to be larger than the largest sequence > number it used on the previous connection > incarnation, > and > > (2) returns to TIME-WAIT state if the SYN turns out to be > an old duplicate." > > This effectively allows new connections to be opened on > sockets in the time-wait state. We've seen this is true for > virtually all operating systems in use today. I think it > would be a good idea to have analogous behavior implemented in tproxy. > > -- > Yuval > > > -----Original Message----- > > From: Balazs Scheidler [mailto:bazsi@balabit.hu] > > Sent: Mon, August 25, 2003 19:55 > > To: Yuval Pemper > > Cc: tproxy(a)lists.balabit.hu; sagig(a)radware.com > > Subject: Re: [tproxy] Question about warning message > > > > > > On Sun, Aug 24, 2003 at 08:08:54AM +0300, Yuval Pemper wrote: > > > While running stress tests on our application, which uses > > the tproxy > > > patch, we see the following warning messages in dmesg: > > > > > > IP_TPROXY: error applying NAT mapping, hooknum=4 .... > > > > > > This warning message comes from the function > > ip_tproxy_setup_nat_bidir in > > > iptable_tproxy.c. It's printed if the result of calling > > ip_nat_setup_info > > > is different than NF_ACCEPT. > > > > > > I'm not sure what this warning means. The > ip_tproxy_setup_nat_bidir > > > function continues normally after the warning is printed > > out. Any help in > > > shedding light on this will be greatly appreciated. > > > > This means that the request NAT mapping to the given foreign > > addr:port pair > > was reserved by another possibly timewaiting connection. I am > > currently > > thinking on adding a feature which would make source port > > allocation more > > liberal. > > > > This usually means that the connection initiated by the proxy > > will go out > > unNATed with the source address it originally bound to. > > > > -- > > Bazsi > > PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD > > 804E C82C 8EB1 > > _______________________________________________ > > tproxy mailing list > > tproxy(a)lists.balabit.hu > > https://lists.balabit.hu/mailman/listinfo/tproxy > > >

1 0

RE: [tproxy] Question about warning message
by Yuval Pemper 26 Aug '03

26 Aug '03

Thanks for your reply. Several remarks: 1. We've seen the phenomenon of packets being sent unNATed. We've also seen that the timeout period affecting this phenomenon is the ip_ct_tcp_timeout_time_wait, which is set to 120 seconds by default. We have changed this value to 1 second to help us deal with this issue. 2. The fact that the connection tracking time-wait timeout affects the amount of time in which unNATed packets are sent out indicates that the connection was in the time-wait state in the connection tracking table. We've seen this is also true for connections whose TCP state was not time-wait, i.e., connections that were passively closed by us. It seems that the connection tracking and TCP state machine differ in this point. Not sure whether this is intended or not. 3. Regarding more liberal source port allocation, RFC 1122 (Requirements for Internet Hosts -- Communication Layers), section 4.2.2.13, states: "When a connection is closed actively, it MUST linger in TIME-WAIT state for a time 2xMSL (Maximum Segment Lifetime). However, it MAY accept a new SYN from the remote TCP to reopen the connection directly from TIME-WAIT state, if it: (1) assigns its initial sequence number for the new connection to be larger than the largest sequence number it used on the previous connection incarnation, and (2) returns to TIME-WAIT state if the SYN turns out to be an old duplicate." This effectively allows new connections to be opened on sockets in the time-wait state. We've seen this is true for virtually all operating systems in use today. I think it would be a good idea to have analogous behavior implemented in tproxy. -- Yuval > -----Original Message----- > From: Balazs Scheidler [mailto:bazsi@balabit.hu] > Sent: Mon, August 25, 2003 19:55 > To: Yuval Pemper > Cc: tproxy(a)lists.balabit.hu; sagig(a)radware.com > Subject: Re: [tproxy] Question about warning message > > > On Sun, Aug 24, 2003 at 08:08:54AM +0300, Yuval Pemper wrote: > > While running stress tests on our application, which uses > the tproxy > > patch, we see the following warning messages in dmesg: > > > > IP_TPROXY: error applying NAT mapping, hooknum=4 .... > > > > This warning message comes from the function > ip_tproxy_setup_nat_bidir in > > iptable_tproxy.c. It's printed if the result of calling > ip_nat_setup_info > > is different than NF_ACCEPT. > > > > I'm not sure what this warning means. The ip_tproxy_setup_nat_bidir > > function continues normally after the warning is printed > out. Any help in > > shedding light on this will be greatly appreciated. > > This means that the request NAT mapping to the given foreign > addr:port pair > was reserved by another possibly timewaiting connection. I am > currently > thinking on adding a feature which would make source port > allocation more > liberal. > > This usually means that the connection initiated by the proxy > will go out > unNATed with the source address it originally bound to. > > -- > Bazsi > PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD > 804E C82C 8EB1 > _______________________________________________ > tproxy mailing list > tproxy(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/tproxy >

1 0

Question about warning message
by Yuval Pemper 25 Aug '03

25 Aug '03

While running stress tests on our application, which uses the tproxy patch, we see the following warning messages in dmesg: IP_TPROXY: error applying NAT mapping, hooknum=4 .... This warning message comes from the function ip_tproxy_setup_nat_bidir in iptable_tproxy.c. It's printed if the result of calling ip_nat_setup_info is different than NF_ACCEPT. I'm not sure what this warning means. The ip_tproxy_setup_nat_bidir function continues normally after the warning is printed out. Any help in shedding light on this will be greatly appreciated. Thanks in advance, -- Yuval Pemper

2 1

how to forward client ip form transperent squid
by Dhruba Raj Bhandari 29 Jul '03

29 Jul '03

Sir, we have some squid server running and they only forward 1 ip address as a source address on behalf of all the hosts behind proxy.They are all configured as a transparent proxy.So I could not do filtering based on ip address before gateway.Is there any solution to make squid to forward original client's ip address. Is tproxy a solution for that. Dhruba

1 0

Re: Difference between DNAT and TPROXY
by jan＠tegtmeier.de 04 Jul '03

04 Jul '03

Balazs Scheidler wrote: > NAT _is_ needed as redirection is implemented using NAT. TCP sessions are > conntracked and this have to be so. Yes, of course. I only worried about the overhead of the double-conntracking and did not think to the end, that it is really needed twice ;-(. Thanks, Jan Tegtmeier

2 1

Connection refused on already accepted connections
by jan＠tegtmeier.de 04 Jul '03

04 Jul '03

Hi again, when using tproxy, all outgoing connections are redirected to the proxy application. This proxy accepts the connection. After the connection is established, the proxy get get the real target with getsockopt. Now it connects to the real target by itself. What happends, if the target rejects the connection, is unreachable or has other problems? The 3way connection handshake between client and proxy is already established and the client must think, that the connection is established with the real target, as tproxy is transparent. The only way I see is to close the connection so the client will get a "connection closed by foreign host". They cannot distinct between different connection problems. Not very nice!? Maybe some applications will have problems with this behaviour? Nice day, Jan Tegtmeier

2 1

Difference between DNAT and TPROXY
by jan＠tegtmeier.de 03 Jul '03

03 Jul '03

Hi, why do I need "iptables -t tproxy -A PREROUTING -j TPROXY --on-port" to replace "iptables -t nat -A PREROUTING -j DNAT --to-dest <localip> --to-port <proxyport>"? Where is the difference? Does the TPROXY-thing use the nat-helper modules of netfilter too? Thanx, Jan Tegtmeier

2 3