Question about rsync and BIG mirror
jamie at shareable.org
Mon Mar 6 18:14:09 GMT 2006
Shachar Shemesh wrote:
> >Hmm. My home directory, on my laptop (a mere 60GB disk), does contain
> >millions of files, and it takes about 20 minutes to build the list on
> >a good day. 100Mbps network, but it's I/O bound not network bound.
> >It looks a lot like the number of files is more significant than the
> >amount of data at this scale.
> In fact, I know of at least one place where they don't use rsync because
> they don't have enough RAM+SWAP to hold the list of files in memory.
That's true when I'm syncing my laptop's home directory (192MB RAM +
256MB swap) to the desktop machine (plenty of RAM).
So I have to split my home directory rsync into two commands, which
copy different parts of it separately.
I suspect the excessive memory usage is due to -H, "preserve hard
links", of which I have many due to mostly-hard-linked Linux kernel
That place you mention wouldn't happen to be using the -H option,
More information about the rsync