tbytes of sync over usb3, exfat, bug & feature request

richard lists at meinsen.net
Sun Aug 23 15:06:52 UTC 2020


currently i  rsynced  several tbytes of files / backups to other/newer drives using fedora 32 x64.
- the data was on ext4 / ntfs drives before.
- the machine has 8GB ram and swap
- different external usb3 drives up to 10tb


- rsync to a exfat drive often halted leaving a defunct rsync process (different data, different drives, always exfat target)
   - i had to sync a 6tb data package by  syncing subdirectories subdirectories step by step
   - for other data packages just kill the defunct process and start again ,  kill and start again, ... finally got it transferred

- the halt doesn't only happen when writing, but also when checking by content "-c"

i'm not sure how to debug this


when doing a comparison "-c" of data on separate external drives rsync reads part of one drive, then reads part oft the other drive, then compares, then reads .. .
usb3 is still slow when it comes to tbytes -> it would be great if reading would be done in parallel when using "-c"  -> could reduce time needed for comparison to nearly 50%



More information about the rsync mailing list