filelist calculation algoritm

[jw schultz]

On Sun, Jan 05, 2003 at 12:44:32PM -0800, Wayne Davison wrote:
> On Sun, Jan 05, 2003 at 11:55:22AM -0800, jw schultz wrote:
> > The first problem is this would flatten things unless you used
> > relative and forced the user's CWD.  That would cause considerable
> > confusion.
> 
> Really?  This is exactly how rsync works now with multiple file names on
> the command-line, so I don't see this as being any more confusing than
> what we already have.  The rule would be you can specify the files on
> the command-line or on stdin (if you use '-' as the only source file).
> Since all names are treated in the same way regardless of where they
> were specified, everything works the same as it did before, only more
> names are now supported per invocation.  I'm thinking that this way is
> more flexible since it allows someone to flatten things if that's what
> they really want to do.

And the effect of that is that users wind looping to produce
scores of rsync sessions to transfer a single list.

> > Secondly, how would you do it when the source location is remote?
> > Many of the users asking for this are doing pulls.
> 
> I mentioned a protocol change that would send the extra file names to
> the other side after rsync starts up.  Currently the send_files()
> routine always sends names from the sending side to the receiving side.
> The new protocol would change that to always send names from the user
> side to the server side when this option was specified.  The user's
> command would look like this:
> 
>     rsync -avR remote:- /foo/bar
> 
> The file list would be read from the local (user) side, of course.  The
> remote command being run by rsync would look like this:
> 
>     ssh remote rsync --server --sender -vlogDtprR . -
> 
> The presence of the '-' as the source would tell us to slurp names
> instead of send them.
> 
> Since the file list is exchanged in total before we do any real work, I
> think this change would actually be really easy to implement.

How many levels down should we allow - to mean "use this
directory as cwd for list"?

	rsync --relative remote::module/- dest/dir

If it can only be "remote:-" then everything would have to
be relative to the user's home directory.
"../../usr/lib/somedir/somefile" anyone?  And/or we
allow absolute paths in the list.  So much for safe-links.

In terms of implimentation i don't think we are that far
apart.  As it stands now we walk the source list.  For each
file/directory we check it against the pattern list prior to
insertion.  At the time of insertion if recursion is turned on
each directory gets a readdir and the contents get the same
test and insert treatment.

You are replacing the source list with stdin.  I'm basically
saying that the list from stdin or from a file would be used
instead of readdir.

Both cases require a protocol bump to support sending the
list for a pull.

The discussion seems to be fruitful but i'd like to see more
participants with other perspectives before i'd bookmark it
as a TODO.

-- 
________________________________________________________________
	J.W. Schultz            Pegasystems Technologies
	email address:		jw at pegasys.ws

		Remember Cernan and Schmitt