Transparent proxying - different approach - NAT on demand.
Hi all. I have some proposition about squid and tproxy. I call it "NAT ON DEMAND". I am not really a good C programmer or kernel hacker, but... AFAIK in early tproxy days a dynamically created SNAT and DNAT rules, was considered but this idea was abandon as unfeasible. My idea is to allow "setting nat on application explicit demand". To create a infrastructure in with userspace application (squid for example) can send a "demand" for source address to witch their outgoing connections should be natted to - on per socket basis.Usage in transparent proxying of all kind. Especially useful for SQUID and realy transparent proxy scenario. The idea is roughly inspired and is analogic to "setting FWMARK from userspace", an old kernel patch that implement ioctl for setting fwmark on local originated packets on per socket basis. See http://oss.sgi.com/archives/netdev/2001-12/msg00053.html 1:applications should be able to explicitly set on per socket basis, on wich IP address they need outgoin connection to be SNATed. Probably add one 32bit item to skbuff and appropriate mechanizm for setting it from userspace 2: change netfilter / iptables to allow "-j SNAT --to-demand" option to witch set a source of the connection to address that application demanded 3: add "-m natdemand" iptables match to find if local originator of this connection demand NAT for it (ie if the additional field in skbuff is not 0x0 ). 4: patch squid to request a nat for its outgoing connections. Only thing that it should to take a client ip address and "demand it" Usage like this in squid conf acl src 192.168.1.0/24 transparent_clients # set ACL for clients network that we need handled transparently nat_on_demand transparent_clients # demand specific SNAT for all outgoing connections (ie connections that fetching content) it made on proxying clients from this network. ie take IP of the squid client and demand SNAT to this IP for outgoing sockets that fetching data for that client on cache miss (sorry my bad english) in iptables: iptables -t nat -I OUTPUT (or POSTROUTING ?) -m natdemand -j SNAT --to-demand # to tell netfilter to SNAT source address of that connection to what application demanded. ie copy address stored in additional skbbuff field
From that point, all the hard work for us is done automatically by netfilter.No need to worry about fancy routing setup etc. No need additional overhead for socket match. Does not interfer with connection tracking as tproxy afaik does. Should work for related (icmp etc) packets as well. Should instantly work for every protocol that have nat helper too. Should be easy to implement and less intrusive (afaik most of tproxy patch is to circumvent some kernel safety measurements and control checks on various places)
Or miss I some point ? PS: excuse my wrong english, that limit severally my ability to express this idea.
Hi, On sze, jan 21, 2009 at 01:42:38 +0100, NTPT wrote:
Hi all.
I have some proposition about squid and tproxy. I call it "NAT ON DEMAND". I am not really a good C programmer or kernel hacker, but...
AFAIK in early tproxy days a dynamically created SNAT and DNAT rules, was considered but this idea was abandon as unfeasible.
My idea is to allow "setting nat on application explicit demand". To create a infrastructure in with userspace application (squid for example) can send a "demand" for source address to witch their outgoing connections should be natted to - on per socket basis.Usage in transparent proxying of all kind. Especially useful for SQUID and realy transparent proxy scenario.
The idea is roughly inspired and is analogic to "setting FWMARK from userspace", an old kernel patch that implement ioctl for setting fwmark on local originated packets on per socket basis. See http://oss.sgi.com/archives/netdev/2001-12/msg00053.html
1:applications should be able to explicitly set on per socket basis, on wich IP address they need outgoin connection to be SNATed. Probably add one 32bit item to skbuff and appropriate mechanizm for setting it from userspace
2: change netfilter / iptables to allow "-j SNAT --to-demand" option to witch set a source of the connection to address that application demanded
3: add "-m natdemand" iptables match to find if local originator of this connection demand NAT for it (ie if the additional field in skbuff is not 0x0 ).
4: patch squid to request a nat for its outgoing connections. Only thing that it should to take a client ip address and "demand it"
Usage like this
in squid conf
acl src 192.168.1.0/24 transparent_clients # set ACL for clients network that we need handled transparently nat_on_demand transparent_clients # demand specific SNAT for all outgoing connections (ie connections that fetching content) it made on proxying clients from this network. ie take IP of the squid client and demand SNAT to this IP for outgoing sockets that fetching data for that client on cache miss (sorry my bad english)
in iptables:
iptables -t nat -I OUTPUT (or POSTROUTING ?) -m natdemand -j SNAT --to-demand # to tell netfilter to SNAT source address of that connection to what application demanded. ie copy address stored in additional skbbuff field
You've just described how tproxy 2 worked, with a slightly different interface.
From that point, all the hard work for us is done automatically by netfilter.No need to worry about fancy routing setup etc. No need additional overhead for socket match. Does not interfer with connection tracking as tproxy afaik does. Should work for related (icmp etc) packets as well. Should instantly work for every protocol that have nat helper too. Should be easy to implement and less intrusive (afaik most of tproxy patch is to circumvent some kernel safety measurements and control checks on various places)
I'd expect the overhead of Netfilter NAT *much* larger than the overhead of the socket match. Also, the need for the 'fancy' routing setup is just because we would have liked to avoid intrusive changes to the kernel. We had a patchset with some changes in the core ipv4 routing code so that absolutely no modifications were necessary to the routing config. We transitioned to the new approach because * it required no changes to routing code, and thus had a better possibility of merging upstream, * it's slightly more flexible (ie. now that the kernel can do policy routing for IPv6 implementing tproxy support for that protocol should be easier. As far as I know the current upstream implementation does not have any problems with (related) ICMP and does not usually interfere with connection tracking. (Unfortunately, because of the way conntrack works you cannot have full connection tracking and do transparent proxying in certain corner cases.) Most of the modifications of the tproxy patch weren't necessary to 'circumvent kernel safety measurements and control checks', but to make it possible to have a TCP endpoint with a non-local IP address in the Linux stack. It *almost* worked out-of-the-box, it's just that the code contained a few checks which interfered with tproxy. This does not mean in any way that the security of the network stack is breached, however: you need to be highly privileged to set the IP_TRANSPARENT flag to make the kernel omit those checks in the IPv4 output path. On the contrary, there were all kinds of complications with tproxy2, all coming from the fact that the local socket table and the connection tracking database of the kernel were out-of-sync (by design). Having been there and done that, I don't think we should go back to that approach... -- KOVACS Krisztian
On Fri, 2009-01-23 at 10:26 +0100, KOVACS Krisztian wrote:
Hi,
On sze, jan 21, 2009 at 01:42:38 +0100, NTPT wrote:
Hi all.
I have some proposition about squid and tproxy. I call it "NAT ON DEMAND". I am not really a good C programmer or kernel hacker, but...
AFAIK in early tproxy days a dynamically created SNAT and DNAT rules, was considered but this idea was abandon as unfeasible.
My idea is to allow "setting nat on application explicit demand". To create a infrastructure in with userspace application (squid for example) can send a "demand" for source address to witch their outgoing connections should be natted to - on per socket basis.Usage in transparent proxying of all kind. Especially useful for SQUID and realy transparent proxy scenario.
The idea is roughly inspired and is analogic to "setting FWMARK from userspace", an old kernel patch that implement ioctl for setting fwmark on local originated packets on per socket basis. See http://oss.sgi.com/archives/netdev/2001-12/msg00053.html
1:applications should be able to explicitly set on per socket basis, on wich IP address they need outgoin connection to be SNATed. Probably add one 32bit item to skbuff and appropriate mechanizm for setting it from userspace
2: change netfilter / iptables to allow "-j SNAT --to-demand" option to witch set a source of the connection to address that application demanded
3: add "-m natdemand" iptables match to find if local originator of this connection demand NAT for it (ie if the additional field in skbuff is not 0x0 ).
4: patch squid to request a nat for its outgoing connections. Only thing that it should to take a client ip address and "demand it"
Usage like this
in squid conf
acl src 192.168.1.0/24 transparent_clients # set ACL for clients network that we need handled transparently nat_on_demand transparent_clients # demand specific SNAT for all outgoing connections (ie connections that fetching content) it made on proxying clients from this network. ie take IP of the squid client and demand SNAT to this IP for outgoing sockets that fetching data for that client on cache miss (sorry my bad english)
in iptables:
iptables -t nat -I OUTPUT (or POSTROUTING ?) -m natdemand -j SNAT --to-demand # to tell netfilter to SNAT source address of that connection to what application demanded. ie copy address stored in additional skbbuff field
You've just described how tproxy 2 worked, with a slightly different interface.
From that point, all the hard work for us is done automatically by netfilter.No need to worry about fancy routing setup etc. No need additional overhead for socket match. Does not interfer with connection tracking as tproxy afaik does. Should work for related (icmp etc) packets as well. Should instantly work for every protocol that have nat helper too. Should be easy to implement and less intrusive (afaik most of tproxy patch is to circumvent some kernel safety measurements and control checks on various places)
I'd expect the overhead of Netfilter NAT *much* larger than the overhead of the socket match. Also, the need for the 'fancy' routing setup is just because we would have liked to avoid intrusive changes to the kernel.
We had a patchset with some changes in the core ipv4 routing code so that absolutely no modifications were necessary to the routing config. We transitioned to the new approach because
* it required no changes to routing code, and thus had a better possibility of merging upstream,
* it's slightly more flexible (ie. now that the kernel can do policy routing for IPv6 implementing tproxy support for that protocol should be easier.
As far as I know the current upstream implementation does not have any problems with (related) ICMP and does not usually interfere with connection tracking. (Unfortunately, because of the way conntrack works you cannot have full connection tracking and do transparent proxying in certain corner cases.)
Most of the modifications of the tproxy patch weren't necessary to 'circumvent kernel safety measurements and control checks', but to make it possible to have a TCP endpoint with a non-local IP address in the Linux stack. It *almost* worked out-of-the-box, it's just that the code contained a few checks which interfered with tproxy. This does not mean in any way that the security of the network stack is breached, however: you need to be highly privileged to set the IP_TRANSPARENT flag to make the kernel omit those checks in the IPv4 output path.
On the contrary, there were all kinds of complications with tproxy2, all coming from the fact that the local socket table and the connection tracking database of the kernel were out-of-sync (by design). Having been there and done that, I don't think we should go back to that approach...
Very well put :) -- Bazsi
participants (3)
-
Balazs Scheidler
-
KOVACS Krisztian
-
NTPT