The rsync daemon as a gateway service?

K S Braunsdorf ksb at sa.fedex.com
Wed Oct 22 23:43:38 EST 2003


In rsyncd.conf I'd like to be able to specify something like:
	[module1]
		gateway 10.168.172.9

or (I think we have to change the protocol a little for this one):
	[module2]
		redirect datasource.my.domain

I've run into this need a few times, and I think it might be useful
enough to add (both of these) to the daemon configuration.  I'm asking
the list to see if either of these might be useful to someone else.


I want to serve a large number of "content modules" to a client
population.  These content modules might be very large and volatile,
and are (by their nature) hosted on many separate servers.  I can
put an rsync daemon on each server, but I can't create a single
host with all the modules on it (too large, and too quickly out of
date).

Presently clients find the content I want via something like:
	each client rsync's down an index
	each client looks up the host with the [module] needed
	each client connects to that data source
which works fine.

That has a few bugs for me, I must
	1) open firewall rules from {all clients} -> {all servers}
	2) build the index files (by polling all the data sources)
	3) [my problem] the network routes from {all server} -> {some clients}
	   are wrong.

I have the rsync daemons up on all the unique servers, I want a "gateway"
rsyncd that dispatches the connections through to the right [module] owner.

When I have all my servers on a non-routable network (or behind a firewall)
I want to be able to tell the "gateway rsyncd" to "pump the connection"
for the client:
	[module1]
		gateway 10.168.172.9

When I have peer rsyncd's I want to be able to redirect clients to
the one with the data, for example on "vulcan" I'd install:
	[vulcan]
		<normal options>
	[terran]
		redirect earth.federation.org

On "earth" I might install:
	[vulcan]
		redirect vulcan.federation.org
	[terran]
		<normal options>


To provide a "read only" service to the Internet I could open a
firewall rule to "rsync.federation.org" and configure:
	[vulcan]
		read only
		gateway vulcan.federation.org
	[terran]
		read only
		gateway earth.federation.org

Adding an optional "port number" on the gateway might be a good idea, and
maybe adding a mapping for the module name (in case they are different).


As a derivative step we could have:
	proxy rsync.founders.org
as a way to poll the "rsync.founders.org" service for a module name,
when we don't have it.  If it has the the module we'll gateway it,
and:
	refer rsync.borg.org
to do that same thing, but redirect to the rsync::site/module if we find it.


I think this allows rsync servers to cluster in a way ftp can't.  I
also think it would be easy to implement.  We would have to check
(in the client) for infinite redirection loops.

--
Thanks for your time,
ksb at sac.fedex.com (KS Braunsdorf)




More information about the rsync mailing list