Copying many sources to different places inside a destination

[Matt McCutchen]

On Mon, 2006-02-27 at 23:18 -0800, Wayne Davison wrote:
> A similar symlink solution could be done on the source side using the
> new --copy-dirlinks option, but only if the nesting of directories
> allowed it to happen (which does not seem to be the case in your
> example).

That would also work if I made symlinks from main-web-content to the
other sources, which isn't a problem.  However, I might have other
dirlinks in the sources that I don't want to follow.  I found a way to
make rsync follow only certain dirlinks, taking advantage of two lucky
behaviors: (1) lstat follows a dirlink specified with a trailing slash,
and (2) rsync filters out duplicates in the file list, favoring
directories over non-directories.  If you like, stop and see if you can
figure out the technique from those hints.

The solution is: use --relative, don't use --copy-dirlinks, and list the
dirlinks to follow as source arguments with trailing slashes (along with
the main source).  rsync recurses inside the main source and does not
follow dirlinks: instead it generates symlink flist entries for them.
Then it recurses inside the target of each specified dirlink and
generates a directory flist entry for the dirlink (and entries for the
files inside).  During flist filtering, rsync drops the symlink entries
for the specified dirlinks.

The ability for a sending rsync to give files file-list names
independent of their real locations in the filesystem would still be
nice even though I can solve my problem without it.  The way to go about
this would probably be to replace file_struct::dir.root with a pointer
to an instance of this structure:
	struct source_arg {
		char *fs_prefix; /* analogue of root */
		char *flist_prefix;
	};
Then, when the sender sends a file-list entry to the receiver, it could
attach the flist_prefix to the beginning of the dirname.  The sender
would also have to create and fill source_arg structures while building
the file list.  The same system that was to be used to allocate id_pairs
could be used for source_args.  This doesn't sound too hard; I might try
to implement it.
-- 
Matt McCutchen
hashproduct at verizon.net
http://hashproduct.metaesthetics.net/