rsync practical size limit?
m-crowder at ti.com
Mon Mar 21 23:14:06 GMT 2005
I've used rsync to transfer multiple terabytes when replacing or
consolidating storage. The practical limit is not the overall size, but
there are a couple that will crop up:
1. This is alluded to in the FAQ, but the relationship between
amount of memory on the hosts and the number of files per sync is
important. Rsync will use ~100 bytes per file on both the source and
destination servers, so if you are moving many millions of files you may
have to break it up into smaller bites - i.e. the "script that writes
other scripts" sort of technique. No real problem here, unless the
files are all in one directory ...
2. If the individual files are much larger than 1GB, rsync will
tend to copy the entire file over again rather than the differences.
This pretty much negates the benefits of rsync. Luckily, most files
that large are database files and they have their own hot migration
It's a VERY useful tool, though.
Texas Instruments, WW Make IT Infrastructure
"Ninety percent of the time things turn out worse than
you thought they would. The other ten percent of the
time you had no right to expect that much."
From: rsync-bounces+m-crowder=ti.com at lists.samba.org
[mailto:rsync-bounces+m-crowder=ti.com at lists.samba.org] On Behalf Of
Sent: Monday, March 21, 2005 2:54 PM
To: rsync at lists.samba.org
Subject: FW: rsync practical size limit?
I'm a heavy user of rsync for what sometimes amounts to more than 175GB
data. Is there a practical size limit for rsync?
To unsubscribe or change options:
Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
-------------- next part --------------
HTML attachment scrubbed and removed
More information about the rsync