socket match can't work with DNAT rules
Hi, all I use the latest tproxy kernel.But I find that, tproxy can't work with DNAT mode. network topology: Web Server(192.168.1.10)----(192.168.1.1)TPROXY Server(202.0.0.1)---(202.0.0.10)Client For TPROXY Server eth0 192.168.1.1 eth1 202.0.0.1 When Client visit TPROXY Server(202.0.0.1) 80 port, we will redirect it to Web Server. There is a DNAT rule for it. iptables -t nat -i eth1 -d 202.0.0.1 -p tcp --dport 80 -j DNAT --to-destination 192.168.1.10 tproxy APP listen on port 50080, and the TPROXY target also set --on-port 50080 for HTTP. Here is the problem: 1. Client send SYN to 202.0.0.1:80 2. TPROXY Server receive it, and TPROXY target will redirect this packet to the socket which is listening on port 50080 3. TPROXY Server send SYN,ACK to the Client 4. Client receive SYN,ACK and send ACK 5. TPROXY Server receive ACK, TPROXY target will redirect this packet to the socket listening on port 50080 6. With DNAT rule, the established socket is 202.0.0.10:port -> 192.168.1.10:80 7. Client send "GET " request to TPROXY Server 8. socket match find this packet doesn't match any socket. For its sip, sport, dip,dport is 202.0.0.10:port -> 202.0.0.1:80, while the established socket is 202.0.0.10:port -> 192.168.1.10:80 So in this case, match can't work correctly for DNAT rules. Anyone has good ideas? Thanks in advance! -- Thanks BR. Wei Dong
On Wed, 2008-12-03 at 21:55 +0800, Dong Wei wrote:
Hi, all
I use the latest tproxy kernel.But I find that, tproxy can't work with DNAT mode.
network topology:
Web Server(192.168.1.10)----(192.168.1.1)TPROXY Server(202.0.0.1)---(202.0.0.10)Client
For TPROXY Server eth0 192.168.1.1 eth1 202.0.0.1 When Client visit TPROXY Server(202.0.0.1) 80 port, we will redirect it to Web Server. There is a DNAT rule for it.
iptables -t nat -i eth1 -d 202.0.0.1 -p tcp --dport 80 -j DNAT --to-destination 192.168.1.10
tproxy APP listen on port 50080, and the TPROXY target also set --on-port 50080 for HTTP. Here is the problem: 1. Client send SYN to 202.0.0.1:80 2. TPROXY Server receive it, and TPROXY target will redirect this packet to the socket which is listening on port 50080 3. TPROXY Server send SYN,ACK to the Client 4. Client receive SYN,ACK and send ACK 5. TPROXY Server receive ACK, TPROXY target will redirect this packet to the socket listening on port 50080 6. With DNAT rule, the established socket is 202.0.0.10:port -> 192.168.1.10:80 7. Client send "GET " request to TPROXY Server 8. socket match find this packet doesn't match any socket. For its sip, sport, dip,dport is 202.0.0.10:port -> 202.0.0.1:80, while the established socket is 202.0.0.10:port -> 192.168.1.10:80
So in this case, match can't work correctly for DNAT rules. Anyone has good ideas?
You cannot use DNAT and tproxy on the same connection. What do you want to achieve? If you want to change the target address of the server side connection, why don't you DNAT the server connection? That should work. -- Bazsi
Hi, Balazs comment 1: "You cannot use DNAT and tproxy on the same connection. What do you want to achieve?" This is a common scenario. Say you have an intermediate compression agent/tunneling agent between the tproxy server and the web server as shown below: Client <--------> tproxy server <---------> compression/tunneling agent1 <=============> compression/tunneling agent2 <------------------> web server In this case, the output from the tproxy server has to be DNATted or policy routed to the compression/tunneling agent. Policy routing is possible if the compression/tunneling agent lies outside the box. In case, if it runs as another process along with the tproxy server, DNAT is the only option, AFAIK. Balazs comment 2: "If you want to change the target address of the server side connection, why don't you DNAT the server connection? That should work." Not able to understand what you exactly mean by "DNAT the server connection". If my understanding is correct, Tproxy association is only for the socket created between client and the tproxy server. If that is the case, why does socket match failure happen for the socket created between tproxy server and DNAT server? Regards, Arun S. 2008/12/3 Balazs Scheidler <bazsi@balabit.hu>
On Wed, 2008-12-03 at 21:55 +0800, Dong Wei wrote:
Hi, all
I use the latest tproxy kernel.But I find that, tproxy can't work with DNAT mode.
network topology:
Web Server(192.168.1.10)----(192.168.1.1)TPROXY Server(202.0.0.1)---(202.0.0.10)Client
For TPROXY Server eth0 192.168.1.1 eth1 202.0.0.1 When Client visit TPROXY Server(202.0.0.1) 80 port, we will redirect it to Web Server. There is a DNAT rule for it.
iptables -t nat -i eth1 -d 202.0.0.1 -p tcp --dport 80 -j DNAT --to-destination 192.168.1.10
tproxy APP listen on port 50080, and the TPROXY target also set --on-port 50080 for HTTP. Here is the problem: 1. Client send SYN to 202.0.0.1:80 2. TPROXY Server receive it, and TPROXY target will redirect this packet to the socket which is listening on port 50080 3. TPROXY Server send SYN,ACK to the Client 4. Client receive SYN,ACK and send ACK 5. TPROXY Server receive ACK, TPROXY target will redirect this packet to the socket listening on port 50080 6. With DNAT rule, the established socket is 202.0.0.10:port -> 192.168.1.10:80 7. Client send "GET " request to TPROXY Server 8. socket match find this packet doesn't match any socket. For its sip, sport, dip,dport is 202.0.0.10:port -> 202.0.0.1:80, while the established socket is 202.0.0.10:port -> 192.168.1.10:80
So in this case, match can't work correctly for DNAT rules. Anyone has good ideas?
You cannot use DNAT and tproxy on the same connection. What do you want to achieve?
If you want to change the target address of the server side connection, why don't you DNAT the server connection? That should work.
-- Bazsi
_______________________________________________ tproxy mailing list tproxy@lists.balabit.hu https://lists.balabit.hu/mailman/listinfo/tproxy
-- Regards, Arun S.
On Tue, 2008-12-16 at 10:46 +0530, Arun Srinivasan wrote:
Hi,
Balazs comment 1: "You cannot use DNAT and tproxy on the same connection. What do you want to achieve?"
This is a common scenario. Say you have an intermediate compression agent/tunneling agent between the tproxy server and the web server as shown below:
Client <--------> tproxy server <---------> compression/tunneling agent1 <=============> compression/tunneling agent2 <------------------> web server
In this case, the output from the tproxy server has to be DNATted or policy routed to the compression/tunneling agent. Policy routing is possible if the compression/tunneling agent lies outside the box. In case, if it runs as another process along with the tproxy server, DNAT is the only option, AFAIK.
Balazs comment 2: "If you want to change the target address of the server side connection, why don't you DNAT the server connection? That should work."
Not able to understand what you exactly mean by "DNAT the server connection".
If my understanding is correct, Tproxy association is only for the socket created between client and the tproxy server. If that is the case, why does socket match failure happen for the socket created between tproxy server and DNAT server?
No, tproxy _may_ operate for both the client->tproxy and the tproxy->server connections, not just between client & tproxy. However, you're right that tproxy does not support redirecting the traffic if the upstream for tproxy is running on the same box, the reason is simple, tproxy also uses routing to direct traffic to a local process, and it is not currently possible to divert outgoing traffic back to the input interface. but never mind. So in the scenario above, what is the added value of using "tproxy" instead of plain, simple NAT. client -> REDIRECT in nat/PREROUTING -> proxy1 -> DNAT in nat/OUTPUT -> compression/tunneling agent1 <-> compression/tunneling agent2 -> webserver the added value of tproxy over using NAT is increased performance (no NAT is required) and solving the problems related to the conntrack table and local IP stack state to get out of sync. -- Bazsi
With the tproxy server as given in my scenario, I have a lot of flexibilities: 1. The high-end caching (tproxy) server can offer thro'put advantages. 2. Original IP of the clients can be retained thro' tproxy support 3. Another IP based traffic identification/metering mechanism can sit between comp/tunneling-agent2 and the web server and monitor the traffic. whereas in the conventional scenario suggested by you, point # 2 and # 3 would fail. Tproxy4 socket is not a secure tunnel like that of IPSec to prevent DNATs from happening. IMHO, I feel that the tproxy-local DNAT issue is a kind of restriction in the name of a new feature. 2008/12/16 Balazs Scheidler <bazsi@balabit.hu>
On Tue, 2008-12-16 at 10:46 +0530, Arun Srinivasan wrote:
Hi,
Balazs comment 1: "You cannot use DNAT and tproxy on the same connection. What do you want to achieve?"
This is a common scenario. Say you have an intermediate compression agent/tunneling agent between the tproxy server and the web server as shown below:
Client <--------> tproxy server <---------> compression/tunneling agent1 <=============> compression/tunneling agent2 <------------------> web server
In this case, the output from the tproxy server has to be DNATted or policy routed to the compression/tunneling agent. Policy routing is possible if the compression/tunneling agent lies outside the box. In case, if it runs as another process along with the tproxy server, DNAT is the only option, AFAIK.
Balazs comment 2: "If you want to change the target address of the server side connection, why don't you DNAT the server connection? That should work."
Not able to understand what you exactly mean by "DNAT the server connection".
If my understanding is correct, Tproxy association is only for the socket created between client and the tproxy server. If that is the case, why does socket match failure happen for the socket created between tproxy server and DNAT server?
No, tproxy _may_ operate for both the client->tproxy and the tproxy->server connections, not just between client & tproxy.
However, you're right that tproxy does not support redirecting the traffic if the upstream for tproxy is running on the same box, the reason is simple, tproxy also uses routing to direct traffic to a local process, and it is not currently possible to divert outgoing traffic back to the input interface. but never mind.
So in the scenario above, what is the added value of using "tproxy" instead of plain, simple NAT.
client -> REDIRECT in nat/PREROUTING -> proxy1 -> DNAT in nat/OUTPUT -> compression/tunneling agent1 <-> compression/tunneling agent2 -> webserver
the added value of tproxy over using NAT is increased performance (no NAT is required) and solving the problems related to the conntrack table and local IP stack state to get out of sync.
-- Bazsi
-- Regards, Arun S.
On Tue, 2008-12-16 at 20:24 +0530, Arun Srinivasan wrote:
With the tproxy server as given in my scenario, I have a lot of flexibilities: 1. The high-end caching (tproxy) server can offer thro'put advantages. 2. Original IP of the clients can be retained thro' tproxy support 3. Another IP based traffic identification/metering mechanism can sit between comp/tunneling-agent2 and the web server and monitor the traffic.
whereas in the conventional scenario suggested by you, point # 2 and # 3 would fail.
Tproxy4 socket is not a secure tunnel like that of IPSec to prevent DNATs from happening.
IMHO, I feel that the tproxy-local DNAT issue is a kind of restriction in the name of a new feature.
true enough, older tproxies did support local OUTPUT rules, but the new one doesn't. it might be possible to support it though with tproxy 4.1 (aka the version of tproxy in linux 2.6.28): * TPROXY rules live in the mangle table * tproxy 4.1 diverts traffic using marks (e.g. it attaches a given mark, and then your routing rules divert these packets to the local IP stack) * you can possibly use the TPROXY target even in mangle/OUTPUT * it might be possible to change the "socket" match in order to look up the local socket based on the source address instead of the destination address (it does not support this at the moment), or use an alternative match to have a grab on the tproxied traffic * the once you have a matching rule, you need to attach the same mark used by the TPROXY rule, which will trigger the same policy routing rule. so technically it should be possible, however I don't really have time to do that right now. I'd be willing to help you, if you are willing to implement it.
2008/12/16 Balazs Scheidler <bazsi@balabit.hu>
On Tue, 2008-12-16 at 10:46 +0530, Arun Srinivasan wrote: > Hi, > > Balazs comment 1: > "You cannot use DNAT and tproxy on the same connection. What do you > want > to achieve?" > > This is a common scenario. Say you have an intermediate compression > agent/tunneling agent between the tproxy server and the web server as > shown below: > > Client <--------> tproxy server <---------> compression/tunneling > agent1 <=============> compression/tunneling agent2 > <------------------> web server > > In this case, the output from the tproxy server has to be DNATted or > policy routed to the compression/tunneling agent. Policy routing is > possible if the compression/tunneling agent lies outside the box. In > case, if it runs as another process along with the tproxy server, DNAT > is the only option, AFAIK. > > Balazs comment 2: > "If you want to change the target address of the server side > connection, > why don't you DNAT the server connection? That should work." > > Not able to understand what you exactly mean by "DNAT the server > connection". > > If my understanding is correct, Tproxy association is only for the > socket created between client and the tproxy server. If that is the > case, why does socket match failure happen for the socket created > between tproxy server and DNAT server?
No, tproxy _may_ operate for both the client->tproxy and the tproxy->server connections, not just between client & tproxy.
However, you're right that tproxy does not support redirecting the traffic if the upstream for tproxy is running on the same box, the reason is simple, tproxy also uses routing to direct traffic to a local process, and it is not currently possible to divert outgoing traffic back to the input interface. but never mind.
So in the scenario above, what is the added value of using "tproxy" instead of plain, simple NAT.
client -> REDIRECT in nat/PREROUTING -> proxy1 -> DNAT in nat/OUTPUT -> compression/tunneling agent1 <-> compression/tunneling agent2 ->
webserver
the added value of tproxy over using NAT is increased performance (no NAT is required) and solving the problems related to the conntrack table and local IP stack state to get out of sync.
-- Bazsi
-- Regards, Arun S.
-- Bazsi
participants (3)
-
Arun Srinivasan
-
Balazs Scheidler
-
Dong Wei