Feature request: Closing destination files in a separate thread
cbiedl at gmx.de
Fri Dec 23 11:38:42 UTC 2016
Adam Nielsen wrote...
> I'm wondering whether it is feasible to have an option that will make
> rsync spawn a separate thread to close files it has created, to avoid
> the main process blocking while the destination files are flushed during
> the close operation?
While your scenario resembles a problem I've been experiencing for years,
I'm not sure whether such a change would help.
> The reason I ask is that it is currently very slow to use rsync on a
> fast locally-mounted network filesystem, as you see the following
If I understand correctly, you transfer from a local file system to a
network (CIFS) one. Does this happen in a local-local scenario as well?
And, is this related to to the size of the files?
My woes were copying many rather small (100k) files unto a USB flash
drive. Once the buffers are filled up, write rate drops to somewhat
one percent of the raw value (like 200 kbyte/sec on a USB 2.0 drive).
My workaround was to switch the file system from ext4 without journal
to f2fs, assuming ext4 puts inode commits into an isolated transaction
(no-barrier didn't help).
Might be a different story but it sounded somewhat familiar.
More information about the rsync