time variable throttling rsync traffic
Eric S. Johansson
esj at harvee.org
Sun Sep 13 20:22:34 MDT 2009
On 9/13/2009 9:20 PM, Matt McCutchen wrote:
> On Sun, 2009-09-13 at 21:02 -0400, Eric S. Johansson wrote:
>> I am using rsync to back up across a VPN. Unfortunately, every so often the
>> home office miscreants drop a big block of data into the backup and that
>> particular backup cycle takes many hours. These same people also complain when
>> net pipe is filled during the day.
>> What I need is an ability to change the bwlimit based on time of day. Any
>> suggestions as to how to do this?
> How about the suggestions you were given on the rsnapshot list?
they're fine as far as they go. I have some serious objections to a fair number
of them, most from operational or understandability perspective. I mostly posed
the question here to see if new ideas would come to the top. the rsnapshot
community is much smaller than that of rsync and I thought it wouldn't be a bad
idea to give the idea a wider audience.
As for the general ideas proposed for using things like firewall rate limiting
and the like, they are all error prone. One of the reasons I liked modifying
bwlimit is that it's easy for any twit to understand and complicated enough for
a smarty to hurt themselves. It is self-contained. It is all within one tool
and there's no way you can hurt or damage anyone else through its use.
The QOS solution is good because if the firewall gets clogged, in theory, rsync
should back off but I would have to wrapper the firewall commands such that a
well-meaning idiot couldn't hurt anyone. This comes back to the all in one
package around some version of rsync.
Unfortunately, the QOS solution only works for the platform you develop it for.
On the other hand, the bwlimit solution works for almost every platform but
doesn't behave well if there are multiple rsync clients talking to one hopes.
The only way around that is to have the clients tell the host and let the host
do the rate limiting for all transactions.
So yeah, all solutions suck in different ways including those we don't know yet.
I'm probably going to go with the QOS solution because it's generally a better
solution but I'm a wee bit nervous about diving into the firewall instructions
because it's been a couple of years. No problems though, I think I'll get it
working with only a few reboots necessary. :-) The hard part will be a good
More information about the rsync