<div dir="ltr"><br><br><div class="gmail_quote"><div dir="ltr">On Tue, Mar 20, 2018 at 5:49 PM Kevin Korb <<a href="mailto:kmk@sanitarium.net">kmk@sanitarium.net</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Nothing there should be preventing incremental indexing. That means it<br>
should start copying as soon as it finds a file that needs to be copied.<br></blockquote><div>Doesn't it tries to create all (empty) directories first? </div><div> </div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
On 03/20/2018 02:33 PM, Bráulio Bhavamitra wrote:<br>
><br>
><br>
> Em seg, 19 de mar de 2018 11:34, Kevin Korb via rsync<br>
> <<a href="mailto:rsync@lists.samba.org" target="_blank">rsync@lists.samba.org</a> <mailto:<a href="mailto:rsync@lists.samba.org" target="_blank">rsync@lists.samba.org</a>>> escreveu:<br>
><br>
> The performance of rsync with a huge number of files is greatly<br>
> determined by every option you are using. So, what is your whole<br>
> command line?<br>
><br>
> rsync -avP /data-old/ /data<br>
><br>
><br>
> On 03/19/2018 09:05 AM, Bráulio Bhavamitra via rsync wrote:<br>
> > Hi all,<br>
> > <br>
> > I'm using rsync 3 to copy all files from one disk to another. The<br>
> files<br>
> > were writen by Minio, an S3 compatible opensource backend.<br>
> ><br>
> > The number of files is dozens of millions, almost each of them within<br>
> > its own directory.<br>
> ><br>
> > Rsync takes a long time, when not several hours, to even start syncing<br>
> > files. I already see a few reasons:<br>
> > - it first create all directories to put files in, that could be done<br>
> > along with the sync<br>
> > - it needs to generate the list of all files before starting, and<br>
> cannot<br>
> > start syncing and keep the list generation in a different thread.<br>
> ><br>
> > Cheers,<br>
> > bráulio<br>
> ><br>
> ><br>
><br>
> --<br>
> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,<br>
> Kevin Korb Phone: (407) 252-6853<br>
> Systems Administrator Internet:<br>
> FutureQuest, Inc. Kevin@FutureQuest.net (work)<br>
> Orlando, Florida <a href="mailto:kmk@sanitarium.net" target="_blank">kmk@sanitarium.net</a><br>
> <mailto:<a href="mailto:kmk@sanitarium.net" target="_blank">kmk@sanitarium.net</a>> (personal)<br>
> Web page: <a href="http://www.sanitarium.net/" rel="noreferrer" target="_blank">http://www.sanitarium.net/</a><br>
> PGP public key available on web site.<br>
> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,<br>
><br>
> --<br>
> Please use reply-all for most replies to avoid omitting the mailing<br>
> list.<br>
> To unsubscribe or change options:<br>
> <a href="https://lists.samba.org/mailman/listinfo/rsync" rel="noreferrer" target="_blank">https://lists.samba.org/mailman/listinfo/rsync</a><br>
> Before posting, read: <a href="http://www.catb.org/~esr/faqs/smart-questions.html" rel="noreferrer" target="_blank">http://www.catb.org/~esr/faqs/smart-questions.html</a><br>
><br>
<br>
--<br>
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,<br>
Kevin Korb Phone: (407) 252-6853<br>
Systems Administrator Internet:<br>
FutureQuest, Inc. Kevin@FutureQuest.net (work)<br>
Orlando, Florida <a href="mailto:kmk@sanitarium.net" target="_blank">kmk@sanitarium.net</a> (personal)<br>
Web page: <a href="http://www.sanitarium.net/" rel="noreferrer" target="_blank">http://www.sanitarium.net/</a><br>
PGP public key available on web site.<br>
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,<br>
<br>
</blockquote></div></div>