rsync oldest files first

[Matt McCutchen]

On Sat, 2009-02-07 at 17:01 -0800, nate wrote:
> The problem is really two fold-  there seems to be some sort
> of issue/bug? with rsync where under certain circumstances it
> will rename one of it's temporary "dot" files to the real
> file name even though it hasn't been successfully copied,

You're using the --partial option, which tells rsync to rename its
temporary file to the real name when it is killed, even if the file is
incomplete.  (The man page could be clearer on this point.)  Why this
didn't happen when you tested rsync manually I can't say.  But try
--partial-dir instead.

> but more importantly I'd like to transfer files that are
> oldest first, either via file includes list or some other
> means. Given the large number of files I would prefer not
> to execute a seperate rsync process for every file..!

Unfortunately, there is no way to make rsync work in non-alphabetical
order.  Some possible approaches:

(1) Put the source files in small groups by date, and run a separate
rsync process for each group.  This will work roughly in date order with
less overhead than a separate rsync per file.

(2) Change the naming convention of the source files so that
alphabetical order is date order.

(3) Use symlinks to make your files accessible at alternate paths for
which alphabetical order is date order.  Details at:

https://bugzilla.samba.org/show_bug.cgi?id=4082#c1

> ssh 10.254.213.203 "mkdir -p /path/to/dest/" &&
> rsync -ae "/usr/bin/hpnssh -v -o TcpRcvBufPoll=yes -o
> NoneEnabled=yes -o NoneSwitch=yes" --timeout=600 --partial
> --log-format="[%p] %t %o %f (%l/%b)"
> --files-from=/home/logrsync/conf/rsync_log_file_list.20090207_124642
> /path/to/source 10.254.213.203:/path/to/dest
> 1>>/home/logrsync/logs/server_name_rsync_log_transfer_20090207_124642.log
> 2>&1

-- 
Matt