On Fri, 2008-01-25 at 15:19 -0800, Pranav Desai wrote:
On Jan 25, 2008 1:19 AM, Balazs Scheidler <bazsi@balabit.hu> wrote:
On Thu, 2008-01-24 at 11:50 -0800, Pranav Desai wrote:
Hello Balazs,
Thanks for the quick reply.
On Jan 24, 2008 1:29 AM, Balazs Scheidler <bazsi@balabit.hu> wrote:
On Wed, 2008-01-23 at 16:40 -0800, Pranav Desai wrote:
Hello All,
I am testing with the example code foreign-tcp-connect on kernel-2.6.20.15/cttproxy-2.0.6 When I run the sample in quick successions, I get this message in /var/log/messages and program returns with errno 17 (File exists).
IP_TPROXY: socket already assigned, reuse=0, e101a8c0:0f27, sr->faddr=c000010a:d007, flags=210000, sr->tv_hashed=1201103313:191693969
Is it ok to add a check to ignore this ?
This means that the local IP:port pair is already present in the tproxy hash table. This is actually not very good, the assignment should be deleted as soon as the socket is closed.
However tracking down these cases proved to be the very problem why we wanted to avoid the current NAT based implementation in the first place. Using this tproxy hash is inherently racy and although we've managed to get it reasonably stable, we encountered similar cases every now and then. And it's very difficult to track down why it happens.
Is it possible to not use the cache ?. If so what kind of performance hit would that be.
This is not a cache. It is inherent in tproxy 2, it contains the mapping between the local IP:port pairs to the foreign IP:port pairs.
This is the mapping used to enact NATing of the outgoing/incoming traffic.
The information in the log message is follows: * SO_REUSEADDR was not set for the socket (I can't remember if it's the old or the new socket) * the local address is 192.168.1.225, port number 9999 or 3879 (can't remember the byte order, it's probably 9999)
it is 9999.
* the requested foreign address is 10.1.0.192, port number 2000 or 533255 * flags is a bitmask of IPT* values, defined at the beginning of the iptable_tproxy file * the sr->tv_hashed value is the exact time of the previous registration
The problem happens probably because the application requests the same local ip:port, and the previous entry was not yet cleaned up, probably because the kernel side of the socket still exists.
Use automatically assigned port numbers and not fixed ones and then it should work.
We are using tproxy in our proxy server application and we already use automatically assigned ports, but its possible that the socket still exists, since this seems to happen when there are a few requests coming in almost simultaneously. Ours is an event based system using epoll.
TProxy cleans up an entry in the tproxy hash whenever an application calls close() on the socket, and the socket structure is dropped from the socket hash in the kernel. There might be some time between close() and the unhashing, but the kernel ensures that it does not reallocate the same port as long as the old one is still in the hashtable.
If your application does not allocate ports by passing '0' to bind, it might step on the same local IP:port pair, so generally to make it reasonably stable you need to ensure that some time passes between port reallocations.
Thanks for your help, we screwed up in our local ports and were using the same ports in some cases.
One last question, is there any performance advantage of using tproxy4 ?
Sure there is, you don't need to load conntrack/nat, doing a NAT mapping requires the following additional steps: for each new connection: * look up in the tproxy hash * setup a nat mapping for each packet: * look up conntrack entry * perform NAT translation -- Bazsi