Feature request: Closing destination files in a separate thread

[Christoph Biedl]

Adam Nielsen wrote...

> I'm wondering whether it is feasible to have an option that will make
> rsync spawn a separate thread to close files it has created, to avoid
> the main process blocking while the destination files are flushed during
> the close operation?

While your scenario resembles a problem I've been experiencing for years,
I'm not sure whether such a change would help.

> The reason I ask is that it is currently very slow to use rsync on a
> fast locally-mounted network filesystem, as you see the following
> behaviour:

If I understand correctly, you transfer from a local file system to a
network (CIFS) one. Does this happen in a local-local scenario as well?
And, is this related to to the size of the files?

My woes were copying many rather small (100k) files unto a USB flash
drive. Once the buffers are filled up, write rate drops to somewhat
one percent of the raw value (like 200 kbyte/sec on a USB 2.0 drive).
My workaround was to switch the file system from ext4 without journal
to f2fs, assuming ext4 puts inode commits into an isolated transaction
(no-barrier didn't help).

Might be a different story but it sounded somewhat familiar.

    Christoph