<html>
<head>
<meta content="text/html; charset=utf-8" http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Hi Alex,<br>
<br>
Thank you very much indeed for that link. I was actually considering
using find -0 together with --files-from, and making find to list
only files that were changed some time ago -- but first, this would
probably break hardlinks, and second, despite most of the files in
question are temporary files, a user can still delete something old
enough right in the middle of an rsync.<br>
<br>
However, looking at rsync problems that we have more closely, I now
realize that most of them are related to time setting issues -- part
of these similar to bug
<a class="moz-txt-link-freetext" href="https://bugzilla.samba.org/show_bug.cgi?id=4977">https://bugzilla.samba.org/show_bug.cgi?id=4977</a> and the like. But I
think these problems can and shall be addressed )<br>
<br>
I will write more on this when I will finish testing.<br>
<br>
Kind regards, George<br>
<br>
<br>
<div class="moz-cite-prefix">On 08/04/17 00:01, Axel Kittenberger
wrote:<br>
</div>
<blockquote
cite="mid:CABg07fsUka0-SU8adH7XoKTCjXMDvmhBQ3R_+E1bnk3D6CVfHA@mail.gmail.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<div dir="ltr">
<div>With this two options on a very live system you may need to
take into account this bug as well I reported a while ago:</div>
<div><br>
</div>
<div><a moz-do-not-send="true"
href="https://protect-au.mimecast.com/s/5vYEBwtZLRbJcm?domain=bugzilla.samba.org">https://bugzilla.samba.org/show_bug.cgi?id=12569</a><br>
</div>
<div><br>
</div>
<div>Due to this I'm currently not using --ignore-missing-args
/ --delete-missing but rather -exclude=* -include=- and a
autocreated hierachical filter rule list on stdin.</div>
<div><br>
</div>
<div>PS: Yes to simply ignore code 24 would be correct. However,
it creates exit code 2 in these cases.</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Apr 7, 2017 at 2:58 PM, Georgy
Fedorov via rsync <span dir="ltr"><<a
moz-do-not-send="true" href="mailto:rsync@lists.samba.org"
target="_blank">rsync@lists.samba.org</a>></span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"> Dear All,<br>
<br>
We sometimes have to replicate large "live" filesystems
with many ( sometimes millions, up to few hundred millions
) files on them. ( Copying actively used files is of
course a bad idea, but it really helps to keep the delta
small, so one final transfer can later save the day. )<br>
<br>
The problem, as one may guess, is that some files may
disappear during the process, so rsync finishes with an
error code 23.<br>
The good news is that since version 3.1.0 rsync has two
options to address this problem: --ignore-missing-args and
--delete-missing-args .<br>
<br>
The bad news, however, is that even with any of these,
when the file disappears, rsync handles the file transfer
properly, but then tries to set times or attributes on
these files nevertheless. That fails with errno 22 (
EINVAL ) and still leads to exit code 23, which is a bit
of an annoyance.<br>
<br>
I am currently trying to come up with a patch to address
this issue, since I'd like to have exit code 0 for either
of --ignore-missing-args or --delete-missing-args, when
the files are, well, missing.<br>
<br>
Probably it is not the right way to address this problem,
but in the same vein as --ignore-missing-args are
implemented, the patch can go as follows:<br>
<a moz-do-not-send="true"
class="m_7728844028767779237moz-txt-link-freetext"
href="https://gist.github.com/anonymous/96ba8bf10f864a93fd9203f75c43ffe9"
target="_blank">https://gist.github.com/<wbr>anonymous/<wbr>96ba8bf10f864a93fd9203f75c43ff<wbr>e9</a>
.<br>
<br>
Basically we need to do two changes: <br>
<br>
(1) in options.c, make sure that the "missing_args" value
is transferred when set ; and <br>
(2) in rsync.c, in the "missing_args" case, replace
FERROR_XFER with something else, since apparently any log
message with logcode FERROR_XFER sets the flag
'got_xfer_error' in log.c, and that finally leads to exit
code RERR_PARTIAL (23), which is what we are trying to
avoid.<br>
<br>
I am currently testing this on fairly big datasets to see
if there's something missing, and will write more when I
see how it goes ( as I said, the data sets are fairly
large ).<br>
<br>
There shall be a better way to address that, but this is
all I can do with a very shallow acquaintance with rsync
source code. Nevertheless, it would be great if that could
be fixed in the trunk one way or another )<br>
<br>
Kind regards, George<span class="HOEnZb"><font
color="#888888"><br>
<br>
<pre class="m_7728844028767779237moz-signature" cols="72">--
George Fedorov
Senior Systems Specialist
Melbourne School of Engineering
The University of Melbourne, Victoria 3010, Australia
phone: *****
email: *****
<a moz-do-not-send="true" class="m_7728844028767779237moz-txt-link-freetext" href="http://www.eng.unimelb.edu.au/" target="_blank">http://www.eng.unimelb.edu.au/</a>
</pre>
</font></span></div>
<br>
--<br>
Please use reply-all for most replies to avoid omitting the
mailing list.<br>
To unsubscribe or change options: <a moz-do-not-send="true"
rel="noreferrer"
href="https://protect-au.mimecast.com/s/gn98BxSeXmW9fJ?domain=lists.samba.org"
target="_blank">https://lists.samba.org/<wbr>mailman/listinfo/rsync</a><br>
Before posting, read: <a moz-do-not-send="true"
rel="noreferrer"
href="https://protect-au.mimecast.com/s/oDLrBgcVgWZDt8?domain=catb.org"
target="_blank">http://www.catb.org/~esr/faqs/<wbr>smart-questions.html</a><br>
</blockquote>
</div>
<br>
</div>
</blockquote>
<br>
</body>
</html>