time variable throttling rsync traffic

[Eric S. Johansson]

On 9/13/2009 9:20 PM, Matt McCutchen wrote:
> On Sun, 2009-09-13 at 21:02 -0400, Eric S. Johansson wrote:
>> I am using rsync to back up across a VPN.  Unfortunately, every so often the
>> home office miscreants drop a big block of data into the backup and that
>> particular backup cycle takes many hours.  These same people also complain when
>> net pipe is filled during the day.
>>
>> What I need is an ability to change the bwlimit based on time of day.  Any
>> suggestions as to how to do this?
>
> How about the suggestions you were given on the rsnapshot list?

they're fine as far as they go.  I have some serious objections to a fair number 
of them, most from operational or understandability perspective.  I mostly posed 
the question here to see if new ideas would come to the top. the rsnapshot 
community is much smaller than that of rsync and I thought it wouldn't be a bad 
idea to give the idea a wider audience.

As for the general ideas proposed for using things like firewall rate limiting 
and the like, they are all error prone.  One of the reasons I liked modifying 
bwlimit is that it's easy for any twit to understand and complicated enough for 
a smarty to hurt themselves.  It is self-contained.  It is all within one tool 
and there's no way you can hurt or damage anyone else through its use.

The QOS solution is good because if the firewall gets clogged, in theory, rsync 
should back off but I would have to wrapper the firewall commands such that a 
well-meaning idiot couldn't hurt anyone.  This comes back to the all in one 
package around some version of rsync.

Unfortunately, the QOS solution only works for the platform you develop it for. 
  On the other hand, the bwlimit solution works for almost every platform but 
doesn't behave well if there are multiple rsync clients talking to one hopes. 
The only way around that is to have the clients tell the host and let the host 
do the rate limiting for all transactions.

So yeah, all solutions suck in different ways including those we don't know yet. 
  I'm probably going to go with the QOS solution because it's generally a better 
solution but I'm a wee bit nervous about diving into the  firewall instructions 
because it's been a couple of years.  No problems though, I think I'll get it 
working with only a few reboots necessary.  :-)  The hard part will be a good 
user interface.