Very slow to start sync with millions of directories and files

[Bráulio Bhavamitra]

Hi all,

I'm using rsync 3 to copy all files from one disk to another. The files
were writen by Minio, an S3 compatible opensource backend.

The number of files is dozens of millions, almost each of them within its
own directory.

Rsync takes a long time, when not several hours, to even start syncing
files. I already see a few reasons:
- it first create all directories to put files in, that could be done along
with the sync
- it needs to generate the list of all files before starting, and cannot
start syncing and keep the list generation in a different thread.

Cheers,
bráulio
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20180319/6bb00855/attachment.html>