Problems with rsync 2.5.1pre1 and hardlinks
dwd at bell-labs.com
Tue Dec 11 06:44:26 EST 2001
On Sun, Dec 09, 2001 at 03:51:29PM +0100, birger at takatukaland.de wrote:
> I got stuck within some weird prob concerning my 2-node linux cluster and
> the synchronisation tool at hand (rsync-2.5.1pre1).
> I have to copy a structure of 70 directories where the data of these
> directories are hardlinked to the data of the 1st directory. Within this
> "orig data" directory, I have about 30.000 files, so the amount of files
> to sync is approx. 2.100.000. The overall size is about 9.2GB.
... [runs out of memory] ...
In case you weren't aware, rsync uses a lot less memory overall if you can
split up the copies into smaller pieces, because it keeps some memory for
every file it touches in a run. In your case, it may be especially
difficult to break up the copy into smaller pieces because of all the
1. Would it be possible to use symlinks instead of hardlinks? That
would give you more flexibility to split things up however you
2. Perhaps you could break it up into ~70 copies, where each time you
give it the first directory that contains the data and another
one that contains one of the hardlinks.
- Dave Dykstra
More information about the rsync