Problems with rsync 2.5.1pre1 and hardlinks
birger at takatukaland.de
birger at takatukaland.de
Tue Dec 11 07:18:33 EST 2001
Dave Dykstra schrieb am Mon, Dec 10, 2001 at 01:44:26PM -0600:
* > I have to copy a structure of 70 directories where the data of these
* > directories are hardlinked to the data of the 1st directory. Within this
* > "orig data" directory, I have about 30.000 files, so the amount of files
* > to sync is approx. 2.100.000. The overall size is about 9.2GB.
*
* ... [runs out of memory] ...
*
* In case you weren't aware, rsync uses a lot less memory overall if you can
* split up the copies into smaller pieces, because it keeps some memory for
* every file it touches in a run. In your case, it may be especially
* difficult to break up the copy into smaller pieces because of all the
* hardlinks.
I thought of that possibility and it won't work exactly of the reason you
described (the hardlink issue).
*
* Ideas:
* 1. Would it be possible to use symlinks instead of hardlinks? That
* would give you more flexibility to split things up however you
* like.
* 2. Perhaps you could break it up into ~70 copies, where each time you
* give it the first directory that contains the data and another
* one that contains one of the hardlinks.
Both alternatives will eat up huge amounts of disk space as the numbers
above suggest. I will therefore consider plugging in more mem/swap before
trying them.
But I thank you for the hint anyway,
- Birger
More information about the rsync
mailing list