Modified --files-from patch

Wayne Davison wayned at users.sourceforge.net
Sun Mar 23 06:21:28 EST 2003


I've been working on an update to my --files-from patch that I think
will soon be ready to be committed to CVS.  See if you agree.

For those that have been following the saga, here's a list of just the
changes since the last patch (for those that haven't, see the next
section):

Changes Since the Last Release:
===============================

 - The --null option was renamed to --from0 and made to effect all the
   --*-from options (--include-from, --exclude-from, and --files-from).
   It has a short-option name of just "-0".

 - Both the include/exclude and --files-from file-reading code now
   allows either CR or LF to indicate a line-ending character (which
   makes rsync work with Unix, DOS, and/or Mac files in all versions).

 - The protocol was tweaked to always send the --files-from data null-
   terminated when it has to send it over the socket.  It also ignores
   empty lines now and the EOF marker (for the socket send) is now
   "////".

 - The --files-from data can now contain full-line comments that start
   with either '#' or ';' (like the include/exclude files).

 - The new options are now documented.

What Is the --files-from Option?
================================

This option allows the user to specify an explicit list of files to
transfer, much like could be done on the command-line, but without an
overly restrictive limit on how many files can be specified.  This can
be much more efficient than doing a recursive directory scan using
include & exclude rules, and enables more complex selection criteria
than can be done by name-matching alone (since the file list is created
by whatever means you desire).

It might help to visualize this new command:

    rsync -a --files-from=/some/list /source remote:/dest

as being roughly equivalent to this old command (note the lack of -r and
the presence of -R -- it's not a mistake):

    cd /source
    rsync -lptgoD -R `cat /some/list` remote:/dest

With these exceptions:

 - Directories mentioned in the "list" file are created on the
   destination (without transferring their contents unless -r is
   specified).

 - Paths in the "list" are sanitized to ensure that they don't refer
   anywhere outside of the default ("/source") dir.

 - The number of files is not limited by the command-line character
   limit.

 - You can specify a list of files separated by nulls (instead of
   newlines) if you add the --from0 option.

 - You can get the /some/list file from the remote end of the connection
   by specifying either "remote:/some/list" or just ":/some/list" (note
   that "remote" must be the same hostname as one end of the transfer).

Where To Get It:
================

    http://www.blorf.net/rsync-files-from.patch

Still To-Do:
============

 - The patch needs testing to ensure that it is working well (I've
   tested all 4 combinations of send direction and list-file source
   location, but more is needed).

 - I need to ensure that the user's input data doesn't contain a line
   of just "////" or the transfer will get messed up (and this may
   well allow an attack on a server daemon).

Enjoy!

..wayne..


More information about the rsync mailing list