moving local files, remote mirroring, and --link-dest awkwardness
Ian! D. Allen
idallen at idallen.ca
Sun Feb 25 04:49:57 GMT 2007
I have some pictures that I unload from the camera into:
and at night I automatically rsync all of dir/ to some remote:dir/
Then days or weeks later I do some local sorting, and I make lots of image
moves of the sort where image dir/tmp/bar goes to to dir/category1/bar,
possibly linked also into dir/category2/bar.
Then rsync runs the next night, notices the "new" image dir/category1/bar
and does the huge image upload all over again, despite the remote system
already having that image in dir/tmp/bar from the previous night.
I need to make rsync use the existing file. Aha! Use link-dest!
But what link-dest?
--link-dest=tmp doesn't work because when rsync can't find category1/bar
on the remote system it looks in the remote --link-dest using
tmp/category1/bar and not using tmp/bar where the file actually is on
the remote system. So rsync thinks it has to copy the whole file again.
My first kludge-approach is to go into the remote dir/tmp and create
hundreds of "ln -s . category1; ln -s . category2; ..." links for all
the dir/* categories. That allows --link-dest=tmp to work; but, it's a
mess and it won't work if I move dir/tmp/bar to dir/category1/subdir/bar.
Any smart people have a better way to handle remote rsync of files that
move from one directory into a set of other directories?
(A second kludge-approach is not to move the files at all, but simply to
link the file from dir/tmp/bar to dir/category1/bar and use the -H
feature to update hard links. That also works; but, then I have a mix
of linked and not-linked files in dir/tmp that I have to sort out.)
| Ian! D. Allen - idallen at idallen.ca - Ottawa, Ontario, Canada
| Home Page: http://idallen.com/ Contact Improv: http://contactimprov.ca/
| College professor (Open Source / Linux) via: http://teaching.idallen.com/
| Defend digital freedom: http://eff.org/ and have fun: http://fools.ca/
More information about the rsync