[tproxy] Question about TCP RST and tproxy

[Stas Grabois]

We have a problem with the following scenario:

1) 	int fd=socket(AF_INET, SOCK_STREAM, 0)
	make fd non-blocking
	bind fd to a local address an port 0
	setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, 1);

2) 	setsockopt(fd, SOL_IP, IP_TPROXY_ASSIGN, {client_ip, client_port});
	setsockopt(fd, SOL_IP, IP_TPROXY_FLAGS, ITP_CONNECT | ITP_ONCE);

3) 	connect(fd, {backend_ip, backend_port}); connect() return -1, errno=EAGAIN;
4)	send(fd, buffer, sizeof(buffer), 0); send returns -1, errno=ECONNREFUSED;
5)	setsockopt(fd, SOL_IP, IP_TPROXY_UNASSIGN); it returns -1, errno=EINVAL;

Our error is the fact that we call send() without checking that connect() 
has failed - it fails
because there is no server and we get RST.

When Linux kernel receives RST - it releases socket's source port, and sets 
sk->num=0;
After that send, which calls sendmsg in af_inet.c, chooses new source port 
via inet_autobind().
inet_autobind() selects new source port, which is used as hash key to 
unassign original
socket, and the setsockopt(IP_TPROXY_UNASSIGN) fails to find the socket.

While we will fix out problem with send()-after-connect(), I think that the 
same problem can
happen on connected socket - when we call send() twice, and get RST between 
the first
and the second send(). Only in this case the second send() will return 
ECONNRESET,
but the source port will be changed anyway.

How difficult is to add handling of RST packets to tproxy module? Some way 
to remove
entries from the tproxy's hash tables on RST...