rsync 3.1.1: --ignore-missing-args / --delete-missing args problem

[Georgy Fedorov]

It is not code 24. RERR_PARTIAL is code 23, which is a bit dangerous to 
ignore.
I may have to fall back to snapshots, but for hierarchical filesystems 
(ZFS in our case) one would have to traverse all .zfs subfolders instead 
of a simple rsync without an -x . This is a unnecessary code 
complication that I was originally trying to avoid.

On 07/04/17 23:12, Kevin Korb via rsync wrote:
> Those options are for handling files/dirs that are specified on the
> actual command line they have nothing to do with files vanishing while
> rsync is running.  The correct solution is to simply ignore exit code 24.
>
> BTW, this is what filesystem snapshots are for.
>
> On 04/07/2017 08:58 AM, Georgy Fedorov via rsync wrote:
>> Dear All,
>>
>> We sometimes have to replicate large "live" filesystems with many (
>> sometimes millions, up to few hundred millions ) files on them. (
>> Copying actively used files is of course a bad idea, but it really helps
>> to keep the delta small, so one final transfer can later save the day. )
>>
>> The problem, as one may guess, is that some files may disappear during
>> the process, so rsync finishes with an error code 23.
>> The good news is that since version 3.1.0 rsync has two options to
>> address this problem: --ignore-missing-args and --delete-missing-args .
>>
>> The bad news, however, is that even with any of these, when the file
>> disappears, rsync handles the file transfer properly, but then tries to
>> set times or attributes on these files nevertheless. That fails with
>> errno 22 ( EINVAL ) and still leads to exit code 23, which is a bit of
>> an annoyance.
>>
>> I am currently trying to come up with a patch to address this issue,
>> since I'd like to have exit code 0 for either of --ignore-missing-args
>> or --delete-missing-args, when the files are, well, missing.
>>
>> Probably it is not the right way to address this problem, but in the
>> same vein as --ignore-missing-args are implemented, the patch can go as
>> follows:
>> https://gist.github.com/anonymous/96ba8bf10f864a93fd9203f75c43ffe9 .
>>
>> Basically we need to do two changes:
>>
>> (1) in options.c, make sure that the "missing_args" value is transferred
>> when set ; and
>> (2) in rsync.c, in the "missing_args" case, replace FERROR_XFER with
>> something else, since apparently any log message with logcode
>> FERROR_XFER sets the flag 'got_xfer_error' in log.c, and that finally
>> leads to exit code RERR_PARTIAL (23), which is what we are trying to avoid.
>>
>> I am currently testing this on fairly big datasets to see if there's
>> something missing, and will write more when I see how it goes ( as I
>> said, the data sets are fairly large ).
>>
>> There shall be a better way to address that, but this is all I can do
>> with a very shallow acquaintance with rsync source code. Nevertheless,
>> it would be great if that could be fixed in the trunk one way or another )
>>
>> Kind regards, George
>>
>> -- 
>> George Fedorov
>> Senior Systems Specialist
>> Melbourne School of Engineering
>> The University of Melbourne, Victoria 3010, Australia
>> phone: *****
>> email: *****
>> http://www.eng.unimelb.edu.au/
>>
>>
>>
>
>

-- 
George Fedorov
Senior Systems Specialist
Melbourne School of Engineering
The University of Melbourne, Victoria 3010, Australia
phone: +61-(0)3-903-56016
email: gfedorov at unimelb.edu.au
http://www.eng.unimelb.edu.au/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20170410/805753e7/attachment.html>