tbytes of sync over usb3, exfat, bug & feature request
richard
lists at meinsen.net
Sun Aug 23 15:06:52 UTC 2020
hi
currently i rsynced several tbytes of files / backups to other/newer drives using fedora 32 x64.
- the data was on ext4 / ntfs drives before.
- the machine has 8GB ram and swap
- different external usb3 drives up to 10tb
bug:
- rsync to a exfat drive often halted leaving a defunct rsync process (different data, different drives, always exfat target)
- i had to sync a 6tb data package by syncing subdirectories subdirectories step by step
- for other data packages just kill the defunct process and start again , kill and start again, ... finally got it transferred
- the halt doesn't only happen when writing, but also when checking by content "-c"
i'm not sure how to debug this
request:
when doing a comparison "-c" of data on separate external drives rsync reads part of one drive, then reads part oft the other drive, then compares, then reads .. .
usb3 is still slow when it comes to tbytes -> it would be great if reading would be done in parallel when using "-c" -> could reduce time needed for comparison to nearly 50%
best
r
More information about the rsync
mailing list