rsync of file list

[Matt McCutchen]

On Wed, 2006-03-01 at 12:04 -0800, lsk wrote:
> ///**** lsk-> Linus in my case the destination server path name and source
> path name that includes SID is different so how you would build the rsync
> syntax using --files-from with this.
> 
> rsync -zv --no-whole-file --stats /d01/app/sourSID/testfile2.dbf 
> tarser:/t01/app/destSID/testfile2.dbf 
> rsync -zv --no-whole-file --stats /d02/app/sourSID/testfile3.dbf 
> tarser:/t02/app/destSID/testfile3.dbf 
> rsync -zv --no-whole-file --stats /d03/app/sourSID/testfile4.dbf 
> tarser:/t03/app/destSID/testfile4.dbf 
> rsync -zv --no-whole-file --stats /d04/app/sourSID/testfile5.dbf 
> tarser:/t04/app/destSID/testfile5.dbf 

I see.  --files-from works when the paths can be split into two parts:
(1) a prefix that is the same for all files but different for the source
and the destination (2) and a suffix that is different for all files but
is the same for the source and the destination.  The d01 and t01 path
components are different for different files _and_ different for the
source and the destination, so a single pass of rsync with --files-from
cannot handle this case without some trickery.

If you rename the destination path components to d01, d02, d03, and d04,
then you can use the same technique as Linus.  Otherwise, you should set
up a symlink mapper on each side to encode the correspondence between
the source and destination paths:
	/mapper/
		testfile2-dir -> /d01/app/sourSID
		testfile3-dir -> /d02/app/sourSID
		testfile4-dir -> /d03/app/sourSID
		testfile5-dir -> /d04/app/sourSID
	tarsus:/mapper/
		testfile2-dir -> /t01/app/sourSID
		testfile3-dir -> /t02/app/sourSID
		testfile4-dir -> /t03/app/sourSID
		testfile5-dir -> /t04/app/sourSID

Now you can use:
	rsync <options> --files-from=list --no-implied-dirs /mapper/ tarsus:/mapper/
where the list contains:
	testfile2-dir/testfile2.dbf
	testfile3-dir/testfile3.dbf
	testfile4-dir/testfile4.dbf
	testfile5-dir/testfile5.dbf
That --no-implied-dirs makes rsync follow the symlinks in the
destination mapper instead of destroying it and trying to create the
files at the same paths as on the source; --keep-dirlinks will have the
same effect.  See the CVS rsync man page for details:
	http://rsync.samba.org/ftp/rsync/nightly/rsync.html

Granted, this is a lot of workaround just to get rsync to copy arbitrary
source paths to arbitrary destination paths.  As I suggested in "Copying
many sources to different places...", it would be nice if rsync
supported this directly via source/destination pairs of --files-from
lines.  Specifically, the sending rsync would read the files off disk
from the source path but send the destination path in the file list.  I
may try to implement this as a patch "arbitrary-pairs.diff" that Wayne
may include in the patches dir or the trunk if he chooses, but I doubt
he'll like the additional word of memory per file-list entry.
-- 
Matt McCutchen
hashproduct at verizon.net
http://hashproduct.metaesthetics.net/