time variable throttling rsync traffic

[Eric S. Johansson]

On 9/14/2009 3:55 PM, Andrew Gideon wrote:

>> If there is one or more bottleneck link in the network (places where
>> traffic feeds from one or more links with aggregate larger capacity
>> into a link with smaller capacity) then it makes sense that this work
>> has to be done on the router that is on the sending side of the
>> bottleneck link(s), not on the receiving side.
>
> I've been assuming that this isn't possible for a couple of reasons.
>
> Since this is a remote backup application, I'm assuming that senders are
> "all over".  That is, there is no one "sending router"; there are
> instead many sending routers.  The only routers common to all these data
> streams would be those close to the receiving server.
>
> Next, most random users have no control over their routing
> infrastructure.

In theory, you could have graph of rsync traffic between multiple hosts.   but 
for right now, let's just consider two hosts. The total bandwidth available 
between any two machines is the lesser of the two bwlimits.  It would need to be 
negotiated periodically as bandwidth limits change over time.  For example, at 8 
a.m. local time, my bandwidth limit my drop to 100 kbytes a second whereas the 
other end may still be available to run full tilt.  I need to drop that link at 
8 a.m. without necessarily restarting the transfer.

It's not a very good model in that you can't take into account and with 
limitations at various chokepoints but, it's better than nothing.

When you expand it to a graph network, you'll need some form of a rendezvous 
point and allocation scheme to allocate bandwidth to all the others.  It would 
get messy because a client to or three hops away could influence the bandwidth 
allocated to a local client.

Unfortunately, I have an idea for experimentation. Tell me how gross is this?

run rsync till a given time deadline, killing off the original program instance 
and then restart with a new bandwidth limit.  I would probably use a small 
program invoking rsync and then sending a signal when "it's Time" then starting 
a new rsync run.

This could only be client-side as server-side couldn't trigger proper restarts I 
don't think.  I'd need to look into that.  My main question would be is there 
any way to tell if rsync has been shut down abruptly by external signal or has 
terminated on its own (with or without errors) ?

I know, this is probably ugly as all get out but, it gives me a chance to 
experiment.