Smoother bandwidth limiting

Jason Philbrook jp at saucer.midcoast.com
Sat Nov 8 01:29:23 EST 2003


> I'm getting more and more convinced that --bwlimit should
> never have gotten into rsync.  Bandwidth management belongs
> at the system level or let it be done with a common
> networking utility instead of at the individual utilities.

I too like --bwlimit. It's not perfect, but it's so easy to adjust
backup/restore speed in the backup program. We use rsync primarily for
offsite backups, so it's great for planning bandwidth use over limited
capacity links. For normal every day use, a slow speed is fine. If there's
an emergency, we can just use rsync --bwlimit option differently to move
files around faster, without having to adjust system level network
scripts, which can vary from system to system.

> Why don't you see about getting it added to openssh.  That
> way it would be usable for more than just rsync.

That's be nice too especial for scp, but I don't use rsync over openssh 
right now.

> If it is going to be changed it should be completely redone.
> Use nanosleep() not the deprecated usleep() nor select() and
> scale the block size according to the size of bwlimit.
> 
> Special case hacks of --bwlimit get my no vote.
> 
> On Tue, Feb 04, 2003 at 03:06:05AM +0200, Mikko Rauhala wrote:
> > Hello
> > 
> > I'm using a cable modem with a slow uplink, and therefore when I want to
> > transfer large amounts of data upstream, I tend to use rsync with
> > --bwlimit. However, the stock rsync seems to send a bit too much data at
> > once for comfort, momentarily blocking my meager upstream enough to
> > bother latency and downstream data transfer (through not getting through
> > enough ack packets when rsync data fills the cabel modem buffer).
> > 
> > I tried a quick kludge, simply limiting the size of a single write()
> > operation so that the write/sleep cycle happens more often, yielding (I
> > hoped) a more steady flow of data, so that the cabel modem buffer
> > wouldn't contain too much data at any point. Through comparing
> > interactive session use before and after the patch, I would have to
> > conclude that this kludge worked in my Debian GNU/Linux (unstable) box.
> > 
> > Now, the point is that I like the kludge and I'd like rsync proper to
> > adopt perhaps a lesser-kludgeish command line option (or something) for
> > this kind of functionality, if you're so inclined. Might be useful to
> > others in similiar circumstances.
> > 
> > Here's my quick kludge (just a one-liner really, thanks to a proper
> > write loop structure), which probably can reduce performance in the
> > general case through using more writes, but has worked nicely for me
> > (against 2.5.5 from Debian's sources; and yes, the "1024" is a magic
> > number):
> > 
> > --- io.c.old    2002-03-22 07:14:44.000000000 +0200
> > +++ io.c        2003-02-04 02:50:14.000000000 +0200
> > @@ -440,7 +440,7 @@
> >                 if (FD_ISSET(fd, &w_fds)) {
> >                         int ret;
> >                         size_t n = len-total;
> > -                       ret = write(fd,buf+total,n);
> > +                       ret = write(fd,buf+total,(n<1024?n:1024));
> >   
> >                         if (ret == -1 && errno == EINTR) {
> >                                 continue;
> > 
> > -- 
> > Mikko Rauhala   - mjr at iki.fi     - <URL:http://www.iki.fi/mjr/>
> > Transhumanist   - WTA member     - <URL:http://www.transhumanism.org/>
> > Singularitarian - SIAI supporter - <URL:http://www.singinst.org/>
> > 
> 
> 
> 
> > -- 
> > To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
> > Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html
> 
> > -- 
> > To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
> > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
> 
> -- 
> ________________________________________________________________
> 	J.W. Schultz            Pegasystems Technologies
> 	email address:		jw at pegasys.ws
> 
> 		Remember Cernan and Schmitt
> -- 
> To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

-- 
/*
Jason Philbrook   |   Midcoast Internet Solutions - Internet Access,
    KB1IOJ        |  Hosting, and TCP-IP Networks for Midcoast Maine
 http://f64.nu/   |             http://www.midcoast.com/
*/



More information about the rsync mailing list