rsync & io

[tim.conway at philips.com]

You're obviously not the first to need this.  That's why the '--bwlimit=' 
option was added.  Set it to however many Kbps you want rsync to use.  I 
usually use 768 going over our wan, to spare some bandwidth on the T1, and 
maybe 10000 over the lan (in my case, all the dasd is nfs).  I suspect 
that it may affect local-to-local transfers as well, but haven't tested 
it, but you can always use localhost:path to make sure.

Tim Conway
tim.conway at philips.com
303.682.4917
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(nnnnnnnnnnnn, 
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), 
".\n" '
"There are some who call me.... Tim?"




Paul Haas <paulh at hamjudo.com>
Sent by: rsync-admin at lists.samba.org
03/21/2002 08:44 AM
Please respond to paulh

 
        To:     rsync at lists.samba.org
Jie Gao <J.Gao at isu.usyd.edu.au>
        cc:     (bcc: Tim Conway/LMT/SC/PHILIPS)
        Subject:        Re: rsync & io
        Classification: 



On Thu, 21 Mar 2002, Jie Gao wrote:

> One problem I have using rsync is that if you have a large dataset,
> rsync will use almost all io, leaving almost none for other processes.

Yup, that's particularly annoying when you use rsync for backups where it
just doesn't matter if it takes one hour or twelve hours.

> I wonder if something can be built into rsync to make it more
> io-friendly so that rsync can coexist with other processes?

I don't think we need to change rsync at all.  Use the "--rsh" parameter
to move the data through a program that limits the bandwidth.  If you're
transferring data between systems, then the bandwidth limiting program can
in turn run the real ssh (or even rsh).  For local transfers, it can run
rsync in daemon mode.

If you're running between Linux systems, and you control the kernel, and
you know more than I do, then you can use QOS, to control bandwidth.  Run
a second ssh server on a nonstandard port, call this the slow-ssh port.
Use QOS to limit the bandwidth used by traffic on that port.  This will
let you limit rsync to say 50% of your bandwidth, even when you've got
more than one rsync process running.

I have not yet configured QOS on my kernel, so I don't know the details.

--
Paul Haas paulh at hamjudo.com


-- 
To unsubscribe or change options: 
http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html