Re: Rsync options –update and –append-verify

[Kevin Korb]

--inplace can prevent rsync from running you out of disk space when
updating a large file.  But you don't want to mix it with --update.
Judging by your other message you think that --update means modiify
files that are already on both ends.  It doesn't.  It only means that
rsync is forbidden  to change files on the target that have a newer
timestamp than is on the source.  If you don't need that then you don't
want --update.

Also, when in doubt, --dry-run.

On 9/11/21 11:51 PM, hancooper wrote:
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Sunday, September 12, 2021 2:34 AM, hancooper via rsync <rsync at lists.samba.org> wrote:
> 
>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>> On Sunday, September 12, 2021 2:03 AM, Kevin Korb kmk at sanitarium.net wrote:
>>
>>> I thought I did elaborate. If it is a problem for you then maybe you
>>> shouldn't be using --update. Or you should let rsync delete incomplete
>>> files upon abort as it does by default.
>>
>> I am using the following
>>
>> rsync -av --inplace --update --log-file=/media/hagbard/hc/snapshot/test1.log /media/hagbard/hc1/snapshot/test1 /media/hagbard/hc/snapshot/
>>
>> I want that if I run the rsync command again, the files already transferred are not resent, thus I used --update.
> 
> Can I re-run without the --update option?  Am I right to say that the files already successfully transferred will not be re-sent again using -av --inplace ?
> 
> What can I do for the files which have been sent but got rsync aborted during transfer?
> 
> 
>>> On 9/11/21 9:29 PM, hancooper wrote:
>>>
>>>> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
>>>> On Saturday, September 11, 2021 11:20 PM, Kevin Korb via rsync rsync at lists.samba.org wrote:
>>>>
>>>>> --archive is all you really need. I actually wish --archive was the
>>>>> default because it is all most people need and with the exception of
>>>>> writing to a FAT filesystem it is almost always needed.
>>>>> --append is for very special cases and should only be used if you really
>>>>> know you need it and why. --append-verify only exists because people
>>>>> seem to think they need --append and get annoyed that it corrupts their
>>>>> files.
>>>>> --update is sometimes helpful however it interacts badly with --partial
>>>>> and --inplace.
>>>>
>>>> That's right. Can you elaborate how --update interacts badly with --partial
>>>> and --inplace?
>>>>
>>>>> Since rsync doesn't set the timestamp of a file until it
>>>>> is finished then any file left incomplete by an aborted rsync will have
>>>>> the timestamp of the abort not the source file. Therefore it will be
>>>>> newer than the source file and unless the source is updated rsync
>>>>> --update will never complete the file.
>>>>
>>>> Is there a solution to this problem and should we consider it a bug?
>>>> I am encountering this, and it's a real problem for me if files are
>>>> not completed.
>>>>
>>>>> On 9/11/21 6:50 PM, hancooper via rsync wrote:
>>>>>
>>>>>> I am struggling to understand exactly what the rsync options --update and --append-verify do.
>>>>>> Doing info rsync gives
>>>>>> -u, --update
>>>>>> This forces rsync to skip any files which exist on the destina‐
>>>>>> tion and have a modified time that is newer than the source
>>>>>> file. (If an existing destination file has a modification time
>>>>>> equal to the source file’s, it will be updated if the sizes are
>>>>>> different.)
>>>>>> --append-verify
>>>>>> This works just like the --append option, but the existing data
>>>>>> on the receiving side is included in the full-file checksum
>>>>>> verification step, which will cause a file to be resent if the
>>>>>> final verification step fails (rsync uses a normal, non-append‐
>>>>>> ing --inplace transfer for the resend).
>>>>>> I am using rsync to transfer directories recursively. There are times where I have to stop the rsync transfer, and resume the transfer a few hours or days later, without affecting the already transferred files at destination.
>>>>>> I also got some files that return errors by rsync, such as
>>>>>> rsync: read errors mapping "/media/hc1/a1-chaos/amvib/IACL-2017-07-19T00:00:00-2017-07-19T23:59:59.mseed": Input/output error (5)
>>>>>> I would like to retry the transfer of these files, at a later time too, without affecting the files that had been transferred successfully.
>>>
>>> --
>>> ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,
>>>
>>> Kevin Korb Phone: (407) 252-6853
>>>
>>> Systems Administrator Internet:
>>>
>>> FutureQuest, Inc. Kevin at FutureQuest.net (work)
>>>
>>> Orlando, Florida kmk at sanitarium.net (personal)
>>>
>>> Web page: https://sanitarium.net/
>>>
>>> PGP public key available on web site.
>>>
>>> ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,
>>
>> --
>>
>> Please use reply-all for most replies to avoid omitting the mailing list.
>> To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
>> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
> 

-- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			https://sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,