filelist calculation algoritm

jw schultz jw at pegasys.ws
Sat Jan 4 20:41:02 EST 2003


On Sat, Jan 04, 2003 at 08:00:20PM +0100, wim delvaux wrote:
> On Saturday 04 January 2003 19:49, Aaron Morris wrote:
> > It has already been suggested in this list, as well as by myself in the
> > rsync wishlist for a new option to specify a file that has a list of
> > files to be transferred.
> 
> 	Perhaps i need to have a look a the sources and see what it takes
> 	to put that option in  (say --files)

This appears to be the most frequently requested feature.

I think it will be reasonable to do we just need correctly
add the list entries to flist with the proper basename.

We keep the source argument on the command-line so that a
top-of-the-tree is specified.  I think we need two types of
file lists.  One specifying subpaths and the other for those
having a shared prefix.  I lean to using two forms to
specify the list rather than a list specifier and a list
mode option.  Regardless of the type of list the argument
count remains the same.  I my examples below is show
reading the list from stdin by using the "-" convention.  I
expect that an actual filename would also be valid.

The one specifying subpaths I'd call --file-list-relative
and each entry would be relative to the top of the tree.

doing
	rsync --file-list-relative - src dest <<EOL"
	file1
	file2
	dir1/file3
	EOL
would actually sync
	src/file1
	src/file2
	src/dir1
	src/dir1/file3
to
	dest/file1
	dest/file2
	dest/dir1
	dest/dir1/file3

In this way you could construct a file list and leave it in
a file for reuse without requiring that the CWD of the rsync
process be always in the same location.  This should
automatically ignore a ./ prefix in the list.

The --file-list-relative would be independent of --relative
It might be that someone else will have a better name for
this.

The shared prefix form would be the default and would
require each entry to be prefixed by the source path.
This form does mean that the contents of the file-list will
determine the required CWD for the rsync process.

doing
	rsync --file-list - src dest <<EOL"
	src/file1
	src/file2
	src/dir1/file3
	EOF
would sync
	src/file1
	src/file2
	src/dir1/file3
to
	dest/file1
	dest/file2
	dest/dir1/file3

This would allow constructions like
	find srcdir | myfilter | rsync --file-list - srcdir destloc

One thing this will need to do is to cope with implied
directories.  What i mean by that is that any intermediate
directories that are not explicitly specified but require
existence should be added to the list.  In the examples
above dir1 is implied so it is synced.  I bring this up
because it would be undesirable to have a directory on the
destination created with less stringent permissions than on
the source. 

For this implicit directory stuff to work recursion should
be disabled.  I would recommend that the --file-list*
options explicitly disable recursion even if it was
otherwise specified.  Recursion disablement will avoid
confusion and allow continued use of combo-options like
-a.

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt



More information about the rsync mailing list