feature request, hardlink progress......

[Paul Slootman]

On Thu 22 Feb 2007, George Georgalis wrote:
> >>
> >>Please upgrade -- hard-link handling is much improved in newer versions.
> >
> >Thanks, turns out there are closer to 500,000 files and 89
> >snapshots (@ ~90% of the files). The process was to install

> > root at target:/root $ rsync --numeric-ids -avHP source:/data/ /data/
> >receiving file list ...
> >ERROR: out of memory in inode_table [receiver]
> >rsync error: error allocating core memory buffers (code 22) at util.c(115) [receiver=2.6.9]
> 
> I added more memory, same problem.
> 
> any recommendations to get around this? will the old version of
> rsync use less memory to make all the hardlinks?

Having done something similar recently (copied an archive with daily
snapshots, unchanged files hardlinked) I can suggest the following:

If your layout is something like this:
/data/YYYYMMDD/ where this is a daily snapshot, you can do:

$ rsync --numeric-ids -avHP source:/data/20060101/ /data/20060101/
(if the first snapshot is 20060101 of course.)

Then:
$ cd /data
$ x=20060101
$ for i in 200601{02,03,...,31}; do
>       rsync -avHP --link-dest=../$x source:/data/$i/ /data/$i/
>       x=$i
> done

If you ensure that the (empty) snapshot directories are already on your
new system, then you can replace the 200601{...} with 2006*, although
you may need to make it skip the first one; I don't know what rsync will
do with a link-dest that is the same as the target :-)


Paul Slootman