[Bug 10575] New: Long Delay for Large Folders Even with Incremental File-List

[samba-bugs at samba.org]

https://bugzilla.samba.org/show_bug.cgi?id=10575

           Summary: Long Delay for Large Folders Even with Incremental
                    File-List
           Product: rsync
           Version: 3.1.0
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P5
         Component: core
        AssignedTo: wayned at samba.org
        ReportedBy: samba at haravikk.com
         QAContact: rsync-qa at samba.org


Okay, so I have a folder I need to backup that unfortunately contains a massive
32,000+ files (it's part of a several hundred gigabytes large disk-image).

However, using rsync to synchronise this folder with a copy on another machine
on my local network takes an incredibly long time, largely thanks to a huge
delay before any files start being sent. It seems that even with the
incremental file-list enabled that rsync is still waiting to scan the entire
folder before any files start being transferred, even though files high up in
the list should be ready to send right away.

Perhaps I'm misunderstanding how the incremental file-list generation works,
but surely the number of files in a folder shouldn't matter for incremental
sending, as it can just send the file-paths in several batches so that the
transfer can start right away? If this is not how it works then please let me
know, and I can file an enhancement instead.

Currently I'm having to use a timestamp based find to pipe files into rsync,
but it's not really how I'd like to do things, as it won't detect deletions;
fortunately those only happen if I manually request the disk image be
compacted, so I can avoid the issue by simply never compacting the image (I
don't need to anyway), but it seems like strange behaviour if incremental
sending can't handle folders with tons of files properly.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.