moving local files, remote mirroring, and --link-dest awkwardness

Ian! D. Allen idallen at idallen.ca
Sun Feb 25 04:49:57 GMT 2007


I have some pictures that I unload from the camera into:

   dir/tmp

and at night I automatically rsync all of dir/ to some remote:dir/

Then days or weeks later I do some local sorting, and I make lots of image
moves of the sort where image dir/tmp/bar goes to to dir/category1/bar,
possibly linked also into dir/category2/bar.

Then rsync runs the next night, notices the "new" image dir/category1/bar
and does the huge image upload all over again, despite the remote system
already having that image in dir/tmp/bar from the previous night.
I need to make rsync use the existing file.  Aha!  Use link-dest!
But what link-dest?

--link-dest=tmp doesn't work because when rsync can't find category1/bar
on the remote system it looks in the remote --link-dest using
tmp/category1/bar and not using tmp/bar where the file actually is on
the remote system.  So rsync thinks it has to copy the whole file again.

My first kludge-approach is to go into the remote dir/tmp and create
hundreds of "ln -s . category1; ln -s . category2; ..." links for all
the dir/* categories.  That allows --link-dest=tmp to work; but, it's a
mess and it won't work if I move dir/tmp/bar to dir/category1/subdir/bar.

Any smart people have a better way to handle remote rsync of files that
move from one directory into a set of other directories?

(A second kludge-approach is not to move the files at all, but simply to
link the file from dir/tmp/bar to dir/category1/bar and use the -H
feature to update hard links.  That also works; but, then I have a mix
of linked and not-linked files in dir/tmp that I have to sort out.)

-- 
| Ian! D. Allen  -  idallen at idallen.ca  -  Ottawa, Ontario, Canada
| Home Page: http://idallen.com/   Contact Improv: http://contactimprov.ca/
| College professor (Open Source / Linux) via: http://teaching.idallen.com/
| Defend digital freedom:  http://eff.org/  and have fun:  http://fools.ca/


More information about the rsync mailing list