Copying a large file from an USB 3 disk (NTFS or EXT4) to another one (UFS/FreeBSD) is a very slow process....

[Kevin Korb]

Rsync is designed to reduce the amount of data transmitted over the 
network.  If rsync isn't networking it can't do that.  However, it still 
uses the same code so it is still using a sender and a receiver rather 
than simply reading and writing as cp does.  Also, rsync forces 
--whole-file because using rsync's algorithm to delta-copy is slower 
than just re-copying a file (especially if one of the local paths is 
really a network mount and double especially if that is the writing end).

On 11/28/21 16:44, Harry Mangalam wrote:
> Can you elaborate on why this is?
> 
> I wrote a parallel rsync wrapper that works very well over networks but 
> is similarly very slow over local disks. I thought it was a bug in my 
> code but didn't get around to tracking it down since my use cases were 
> all network/ parallel file systems.
> Harry
> 
> On Sun, Nov 28, 2021, 11:43 AM Kevin Korb via rsync 
> <rsync at lists.samba.org <mailto:rsync at lists.samba.org>> wrote:
> 
>     rsync is terribly slow at local copies.  Also, it doesn't do its normal
>     optimizing (see --whole-file).  Just use cp.
> 
>     On 11/28/21 13:38, Mario Marietto via rsync wrote:
>      > Hello to everyone.
>      >
>      > I'm copying a large file from a NTFS formatted disk to another
>      > one,UFS/FreeBSD disk,both are removable disks attached to the USB 3
>      > port. The file is 200 GB large and it is copied very slowly. Why
>     it is
>      > so slow ? I don't know where to store my virtual machines. I
>     tried to
>      > save them on the ext4 disk because I wanted to share them easily
>     between
>      > Linux and FreeBSD but I've realized that when I mount the disk in
>      > FreeBSD after some time it corrupts. I tried to store it on the NTFS
>      > disk but it happens the same. So,now I'm on FreeBSD and I'm
>     copying them
>      > to a dedicated UFS/FreeBSD style disk,but as I said,the speed is
>     very
>      > slow. How can I increase the speed ?. Actually I'm using this
>      > command,because I want to resume the uploading if it breaks at
>     some point :
>      >
>      >
>      > root at marietto:/mnt/da3p2/bhyve/Ubuntu # rsync -avAXEWSlHh
>      > /mnt/da0p1/Backups/OS/bhyve/Ubuntu/im* . --no-compress
>     --info=progress2
>      >
>      > sending incremental file list
>      > impish-cuda-11-4-nvidia-470.img
>      >
>      > 2.13M 0% 9.49kB/s 6284:55:38
>      >
>      >
>      > and : where do you save large files ? what's the procedure that
>     you use
>      > to copy large files with a decent speed ? Unfortunately under
>     Linux is
>      > not safe to use a RW ufs disk access. So,I'm out of solutions.
>      >
>      > -
>      > Mario.
>      >
> 
>     -- 
>     ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
>              Kevin Korb                      Phone:    (407) 252-6853
>              Systems Administrator           Internet:
>              FutureQuest, Inc.               Kevin at FutureQuest.net  (work)
>              Orlando, Florida kmk at sanitarium.net
>     <mailto:kmk at sanitarium.net> (personal)
>              Web page: https://sanitarium.net/ <https://sanitarium.net/>
>              PGP public key available on web site.
>     ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
> 
>     -- 
>     Please use reply-all for most replies to avoid omitting the mailing
>     list.
>     To unsubscribe or change options:
>     https://lists.samba.org/mailman/listinfo/rsync
>     <https://lists.samba.org/mailman/listinfo/rsync>
>     Before posting, read:
>     http://www.catb.org/~esr/faqs/smart-questions.html
>     <http://www.catb.org/~esr/faqs/smart-questions.html>
> 

-- 
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
	Kevin Korb			Phone:    (407) 252-6853
	Systems Administrator		Internet:
	FutureQuest, Inc.		Kevin at FutureQuest.net  (work)
	Orlando, Florida		kmk at sanitarium.net (personal)
	Web page:			https://sanitarium.net/
	PGP public key available on web site.
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,