August 2003 - tproxy - lists.balabit.hu

RE: [tproxy] Question about warning message
by Yuval Pemper 27 Aug '03

27 Aug '03

Further to the email below, we've found an interim solution that fits our requirements: 1. We don't want any unNATed tproxy packets to be sent by our application. 2. We want NATed tproxy packets to be sent out ASAP. Our solution is to drop packets for which the call to ip_tproxy_setup_nat returns NF_DROP. In addition, we reduce the connection tracking time-wait timeout to 1 second and the initial RTO for SYN packets to 1.05 seconds (we took a spare of 50 msec). The simple patch for accomplishing the above (not including changing the connection tracking time-wait timeout) is given below. Obviously, a better solution would be to allow setting up tproxy NAT for conflicting time-wait connections, as described in my previous email. -- Yuval diff -Nru linux.org/include/net/tcp.h linux/include/net/tcp.h --- linux.org/include/net/tcp.h 2003-08-11 17:53:55.000000000 +0300 +++ linux/include/net/tcp.h 2003-08-27 09:18:22.000000000 +0300 @@ -334,6 +334,7 @@ #define TCP_RTO_MAX ((unsigned)(120*HZ)) #define TCP_RTO_MIN ((unsigned)(HZ/5)) #define TCP_TIMEOUT_INIT ((unsigned)(3*HZ)) /* RFC 1122 initial RTO value */ +#define TCP_SYN_TIMEOUT_INIT ((unsigned)(1.05*HZ)) /* SYN RETRANSMIT TIMEOUT */ #define TCP_RESOURCE_PROBE_INTERVAL ((unsigned)(HZ/2U)) /* Maximal interval between probes * for local resources. diff -Nru linux.org/net/ipv4/netfilter/iptable_tproxy.c linux/net/ipv4/netfilter/iptable_tproxy.c --- linux.org/net/ipv4/netfilter/iptable_tproxy.c 2003-08-11 17:53:24.000000000 +0300 +++ linux/net/ipv4/netfilter/iptable_tproxy.c 2003-08-27 09:16:40.000000000 +0300 @@ -581,7 +581,7 @@ * necessary NAT function */ - ip_tproxy_setup_nat(*pskb, hooknum, sr, !(sr->flags & TF_UNIDIR)); + verdict = ip_tproxy_setup_nat(*pskb, hooknum, sr, !(sr->flags & TF_UNIDIR)); if (sr->flags & TF_NAT_ONCE) { /* we were to be applied only once */ @@ -625,7 +625,7 @@ sr.lport = lport; sr.proto = iph->protocol; DEBUGP(KERN_DEBUG "IP_TPROXY: performing redirect to %08x:%04x\n", sr.laddr, sr.lport); - ip_tproxy_setup_nat(*pskb, hooknum, &sr, 0); + verdict = ip_tproxy_setup_nat(*pskb, hooknum, &sr, 0); } } } diff -Nru linux.org/net/ipv4/tcp_output.c linux/net/ipv4/tcp_output.c --- linux.org/net/ipv4/tcp_output.c 2003-01-31 14:13:49.000000000 +0200 +++ linux/net/ipv4/tcp_output.c 2003-08-27 09:17:02.000000000 +0300 @@ -1201,7 +1201,7 @@ tp->rcv_wup = 0; tp->copied_seq = 0; - tp->rto = TCP_TIMEOUT_INIT; + tp->rto = TCP_SYN_TIMEOUT_INIT; tp->retransmits = 0; tcp_clear_retrans(tp); } -- Yuval > -----Original Message----- > From: Yuval Pemper > Sent: Tue, August 26, 2003 8:43 > To: 'Balazs Scheidler' > Cc: tproxy(a)lists.balabit.hu; Sagi Grabois > Subject: RE: [tproxy] Question about warning message > > > Thanks for your reply. Several remarks: > > 1. We've seen the phenomenon of packets being sent unNATed. > We've also seen that the timeout period affecting this > phenomenon is the ip_ct_tcp_timeout_time_wait, which is set > to 120 seconds by default. We have changed this value to 1 > second to help us deal with this issue. > > 2. The fact that the connection tracking time-wait timeout > affects the amount of time in which unNATed packets are sent > out indicates that the connection was in the time-wait state > in the connection tracking table. We've seen this is also > true for connections whose TCP state was not time-wait, i.e., > connections that were passively closed by us. It seems that > the connection tracking and TCP state machine differ in this > point. Not sure whether this is intended or not. > > 3. Regarding more liberal source port allocation, RFC 1122 > (Requirements for Internet Hosts -- Communication Layers), > section 4.2.2.13, states: > > "When a connection is closed actively, it MUST linger in > TIME-WAIT state for a time 2xMSL (Maximum Segment Lifetime). > However, it MAY accept a new SYN from the remote TCP to > reopen the connection directly from TIME-WAIT state, if it: > > (1) assigns its initial sequence number for the new > connection to be larger than the largest sequence > number it used on the previous connection > incarnation, > and > > (2) returns to TIME-WAIT state if the SYN turns out to be > an old duplicate." > > This effectively allows new connections to be opened on > sockets in the time-wait state. We've seen this is true for > virtually all operating systems in use today. I think it > would be a good idea to have analogous behavior implemented in tproxy. > > -- > Yuval > > > -----Original Message----- > > From: Balazs Scheidler [mailto:bazsi@balabit.hu] > > Sent: Mon, August 25, 2003 19:55 > > To: Yuval Pemper > > Cc: tproxy(a)lists.balabit.hu; sagig(a)radware.com > > Subject: Re: [tproxy] Question about warning message > > > > > > On Sun, Aug 24, 2003 at 08:08:54AM +0300, Yuval Pemper wrote: > > > While running stress tests on our application, which uses > > the tproxy > > > patch, we see the following warning messages in dmesg: > > > > > > IP_TPROXY: error applying NAT mapping, hooknum=4 .... > > > > > > This warning message comes from the function > > ip_tproxy_setup_nat_bidir in > > > iptable_tproxy.c. It's printed if the result of calling > > ip_nat_setup_info > > > is different than NF_ACCEPT. > > > > > > I'm not sure what this warning means. The > ip_tproxy_setup_nat_bidir > > > function continues normally after the warning is printed > > out. Any help in > > > shedding light on this will be greatly appreciated. > > > > This means that the request NAT mapping to the given foreign > > addr:port pair > > was reserved by another possibly timewaiting connection. I am > > currently > > thinking on adding a feature which would make source port > > allocation more > > liberal. > > > > This usually means that the connection initiated by the proxy > > will go out > > unNATed with the source address it originally bound to. > > > > -- > > Bazsi > > PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD > > 804E C82C 8EB1 > > _______________________________________________ > > tproxy mailing list > > tproxy(a)lists.balabit.hu > > https://lists.balabit.hu/mailman/listinfo/tproxy > > >

1 0

RE: [tproxy] Question about warning message
by Yuval Pemper 26 Aug '03

26 Aug '03

Thanks for your reply. Several remarks: 1. We've seen the phenomenon of packets being sent unNATed. We've also seen that the timeout period affecting this phenomenon is the ip_ct_tcp_timeout_time_wait, which is set to 120 seconds by default. We have changed this value to 1 second to help us deal with this issue. 2. The fact that the connection tracking time-wait timeout affects the amount of time in which unNATed packets are sent out indicates that the connection was in the time-wait state in the connection tracking table. We've seen this is also true for connections whose TCP state was not time-wait, i.e., connections that were passively closed by us. It seems that the connection tracking and TCP state machine differ in this point. Not sure whether this is intended or not. 3. Regarding more liberal source port allocation, RFC 1122 (Requirements for Internet Hosts -- Communication Layers), section 4.2.2.13, states: "When a connection is closed actively, it MUST linger in TIME-WAIT state for a time 2xMSL (Maximum Segment Lifetime). However, it MAY accept a new SYN from the remote TCP to reopen the connection directly from TIME-WAIT state, if it: (1) assigns its initial sequence number for the new connection to be larger than the largest sequence number it used on the previous connection incarnation, and (2) returns to TIME-WAIT state if the SYN turns out to be an old duplicate." This effectively allows new connections to be opened on sockets in the time-wait state. We've seen this is true for virtually all operating systems in use today. I think it would be a good idea to have analogous behavior implemented in tproxy. -- Yuval > -----Original Message----- > From: Balazs Scheidler [mailto:bazsi@balabit.hu] > Sent: Mon, August 25, 2003 19:55 > To: Yuval Pemper > Cc: tproxy(a)lists.balabit.hu; sagig(a)radware.com > Subject: Re: [tproxy] Question about warning message > > > On Sun, Aug 24, 2003 at 08:08:54AM +0300, Yuval Pemper wrote: > > While running stress tests on our application, which uses > the tproxy > > patch, we see the following warning messages in dmesg: > > > > IP_TPROXY: error applying NAT mapping, hooknum=4 .... > > > > This warning message comes from the function > ip_tproxy_setup_nat_bidir in > > iptable_tproxy.c. It's printed if the result of calling > ip_nat_setup_info > > is different than NF_ACCEPT. > > > > I'm not sure what this warning means. The ip_tproxy_setup_nat_bidir > > function continues normally after the warning is printed > out. Any help in > > shedding light on this will be greatly appreciated. > > This means that the request NAT mapping to the given foreign > addr:port pair > was reserved by another possibly timewaiting connection. I am > currently > thinking on adding a feature which would make source port > allocation more > liberal. > > This usually means that the connection initiated by the proxy > will go out > unNATed with the source address it originally bound to. > > -- > Bazsi > PGP info: KeyID 9AF8D0A9 Fingerprint CD27 CFB0 802C 0944 9CFD > 804E C82C 8EB1 > _______________________________________________ > tproxy mailing list > tproxy(a)lists.balabit.hu > https://lists.balabit.hu/mailman/listinfo/tproxy >

1 0

Question about warning message
by Yuval Pemper 25 Aug '03

25 Aug '03

While running stress tests on our application, which uses the tproxy patch, we see the following warning messages in dmesg: IP_TPROXY: error applying NAT mapping, hooknum=4 .... This warning message comes from the function ip_tproxy_setup_nat_bidir in iptable_tproxy.c. It's printed if the result of calling ip_nat_setup_info is different than NF_ACCEPT. I'm not sure what this warning means. The ip_tproxy_setup_nat_bidir function continues normally after the warning is printed out. Any help in shedding light on this will be greatly appreciated. Thanks in advance, -- Yuval Pemper

2 1