Questions and comments regarding --remove-sent-files (Was: New
eharris at puremagic.com
Tue Sep 12 19:05:10 GMT 2006
I've looked back through my mailing list archives, and seen a few messages
touching on the same things I wanted to mention, but I figured it might be
better to recap, since most of them were sent more than a year ago.
I have recently started using the --remove-sent-files option, and have
noticed a couple of warts. I'm using it to transfer (move really) gigabyte
and larger sized files over a fairly slow connection (--bwlimit=10) and with
keeping of partial files (--partial) to minimize transfer time in the event
of connection problems.
Because the individual files may take a day or more each to transfer, rsync
interruptions are not uncommon, and I've had several instances where the
first run of a transfer aborted in the middle of the non-first file.
Although rsync had successfully sent one or more files before losing the
connection or being aborted, it doesn't appear to delete the files until a
successful end of the whole rsync. A later restart of the rsync sees that
some of the files already exist on the destination and need no update, and
those files get left on the sending side when they shouldn't.
So, I agree with the parent message that either --remove-sent-files should
delete the files immediately after they are successfully sent, or a new
option should be added (--move maybe?) that does it that way.
I saw a followup mailing list message from Wayne that suggested adding the
-I option to cause the desired behavior, and that looks like it would be a
good workaround. Maybe all that is needed is to make a new --move option be
an alias for --remove-sent-files and --ignore-times. Would this be a fairly
The other issue I wanted to touch on was also mentioned on the mailing list,
and was how to guard against the possibility that files on the sending side
might have been modified during the transfer (which for me sometimes takes a
day or more), and for rsync to realize this and avoid deleting the file and
losing those changes. I know this one is a more difficult problem, but I
just wanted to see if there might be an easy solution.
Wayne, thanks for all your work!
On Fri, 22 Apr 2005, Wayne Davison wrote:
> [It appears I missed this message back in February -- ouch.]
> On Sat, Feb 19, 2005 at 08:53:32PM -0500, Andrew Gideon wrote:
>> FWIW: In the manner I can envision using this, it makes more sense to
>> delete the source as long as the destination file is valid, whether
>> that file moved during this execution or not. This provides a "mv"
>> function that's safe against a failure.
> That is an interesting point. The current option allows you to have
> identical files that didn't get transferred, and thus don't get removed,
> but that does mean that if the transfer gets interrupted it might do the
> wrong thing with a file.
> I'll contemplate what to do going forward since the --remove-sent-files
> option was already released. Perhaps a --remove-source-files option
> should be added that works as you suggested.
> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
More information about the rsync