On Tue, 2008-10-14 at 09:15 -0400, Ravi Valmikam wrote:

This is the setup:

There are 3 PCs.

1. Client 2. A custom TCP proxy based on 2.6.24 + tproxy 3. Server

All pretty high end machines.

Test sequence:

Step A. I send a data packet on already established TCP packet. Size is an MTU from machine 1. Step B. Machine 2 receives it and proxies it. Step C. Machine 3 receives it and echoes it back.

We measure the on-the-wire time in and out of machine (2).

In pure ipforwarding mode on machine 2, we see 2-3 ms latency.

In tproxy mode on machine 2, we see 5-6 ms latency (additional 3ms).

I hope that clarifies the scenario.

Well proxying in userspace definitely adds overheads over simple packet forwarding: * instead of simple packet forwarding, the packets are processed by the local TCP stack and queued to a socket buffer * the userspace application wakes up if there's traffic, since this is in userspace, it takes some time (remember packet forwarding is all performed in softirq context, whereas the userspace proxies have to react on data stored in a socket buffer by the same softirq contex) * the userspace application copies data from the receiving socket buffer and puts that data in the target socket buffer, that copies the data twice (whereas IP forwarding does not copy packets, except for maybe in the ethernet driver, but that applies to TCP proxies too) So there's plenty of overhead, but that's the price of the increased flexibility you pay for the flexibility that you can do in userspace/proxies. So the latency is not tproxy specific, it's proxying specific. If you want to measure tproxy performance, you should measure the difference between a simple non-transparent proxy and one where tproxy redirects traffic. I doubt there'd be too much of a difference. -- Bazsi