specifying a list of files to transfer

[Wayne Davison]

> On Wed, Jan 15, 2003 at 02:49:08PM -0800, jw schultz wrote:
> > You seem to see the files-from as a way of replacing command-line
> > args where i see it as a way of replacing the tree scan.

I actually think of it as both, since I also consider the command-line
as a way of replacing the tree scan.  I think that it is fairly easy to
explain how --files-from works if you explain it in terms of how it is
much like specifying the names on the command-line (and explain what is
different about it).  One thing that I'm hoping to avoid is arbitrary
limits on what the mode can do -- I'd like to see it be an easy way to
specify exactly what files to send, and also as a way to extend the size
of the command-line.

On Thu, Jan 16, 2003 at 02:52:38PM -0600, Dave Dykstra wrote:
> I would rather have the '-r' option ignored when --files-from is in
> effect.

I wouldn't want that as a hard limit.  It would be better to say that
-a doesn't imply -r when --files-from is used, but the user can still
manually specify -r if they want to.  It would be easy to implement this
in a way that would _not_ require any particular order to the options on
the command-line (which I agree with JW would be a very bad idea).

> If people leave out the directories, missing parent directories should
> be automatically created.

(I assume you're talking about when using -R, which is not currently on
by default.)  I believe that we do need an auto-creation mode and also a
way to optimize the transfer to avoid this (since it results in a lot of
extra directory checks that can slow things down when we know that they
aren't needed).  Which one is the default is the current question.  I'm
currently leaning toward going back to sending the implied dirs by
default, and having an option for people to optimize the transfer (which
would allow it to be used with the normal -R mode even when --from-files
was not used).

> As it is now, if somebody just does "find . -print | rsync -a
> --files-from- ..." are they going to get repeated files because the
> directories are listed?

Rsync would weed out the duplicates, but if -a implied -r in this
context then the presence of the directories would cause rsync to
recurse through all the directory content and thus make this a
horrible thing to do.  A couple alternatives:

  find . -print | rsync -lptgoDR --files-from=- . remote:/dest
  find . -type f -print | rsync -aR --files-from=- . remote:/dest

> Also, if the transfer is being sent from the remote side, the file names
> are all getting sent over to the remote side first for --files-from and
> then sent back as part of the normal protocol, right?  I had hoped we'd
> be able to avoid that round trip because the list could get long.

I don't see a way to avoid this and still allow things like the creation
of the implied directories you mentioned (which must be sent as separate
entries in the current protocol).  I think it's probably best to leave
the current file+info send process alone.

..wayne..