Copying a large file from an USB 3 disk (NTFS or EXT4) to another one (UFS/FreeBSD) is a very slow process....

[Harry Mangalam]

Ok, thanks for that explanation.
H

On Sun, Nov 28, 2021, 1:59 PM Kevin Korb <kmk at sanitarium.net> wrote:

> Rsync is designed to reduce the amount of data transmitted over the
> network.  If rsync isn't networking it can't do that.  However, it still
> uses the same code so it is still using a sender and a receiver rather
> than simply reading and writing as cp does.  Also, rsync forces
> --whole-file because using rsync's algorithm to delta-copy is slower
> than just re-copying a file (especially if one of the local paths is
> really a network mount and double especially if that is the writing end).
>
> On 11/28/21 16:44, Harry Mangalam wrote:
> > Can you elaborate on why this is?
> >
> > I wrote a parallel rsync wrapper that works very well over networks but
> > is similarly very slow over local disks. I thought it was a bug in my
> > code but didn't get around to tracking it down since my use cases were
> > all network/ parallel file systems.
> > Harry
> >
> > On Sun, Nov 28, 2021, 11:43 AM Kevin Korb via rsync
> > <rsync at lists.samba.org <mailto:rsync at lists.samba.org>> wrote:
> >
> >     rsync is terribly slow at local copies.  Also, it doesn't do its
> normal
> >     optimizing (see --whole-file).  Just use cp.
> >
> >     On 11/28/21 13:38, Mario Marietto via rsync wrote:
> >      > Hello to everyone.
> >      >
> >      > I'm copying a large file from a NTFS formatted disk to another
> >      > one,UFS/FreeBSD disk,both are removable disks attached to the USB
> 3
> >      > port. The file is 200 GB large and it is copied very slowly. Why
> >     it is
> >      > so slow ? I don't know where to store my virtual machines. I
> >     tried to
> >      > save them on the ext4 disk because I wanted to share them easily
> >     between
> >      > Linux and FreeBSD but I've realized that when I mount the disk in
> >      > FreeBSD after some time it corrupts. I tried to store it on the
> NTFS
> >      > disk but it happens the same. So,now I'm on FreeBSD and I'm
> >     copying them
> >      > to a dedicated UFS/FreeBSD style disk,but as I said,the speed is
> >     very
> >      > slow. How can I increase the speed ?. Actually I'm using this
> >      > command,because I want to resume the uploading if it breaks at
> >     some point :
> >      >
> >      >
> >      > root at marietto:/mnt/da3p2/bhyve/Ubuntu # rsync -avAXEWSlHh
> >      > /mnt/da0p1/Backups/OS/bhyve/Ubuntu/im* . --no-compress
> >     --info=progress2
> >      >
> >      > sending incremental file list
> >      > impish-cuda-11-4-nvidia-470.img
> >      >
> >      > 2.13M 0% 9.49kB/s 6284:55:38
> >      >
> >      >
> >      > and : where do you save large files ? what's the procedure that
> >     you use
> >      > to copy large files with a decent speed ? Unfortunately under
> >     Linux is
> >      > not safe to use a RW ufs disk access. So,I'm out of solutions.
> >      >
> >      > -
> >      > Mario.
> >      >
> >
> >     --
> >
>  ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
> >              Kevin Korb                      Phone:    (407) 252-6853
> >              Systems Administrator           Internet:
> >              FutureQuest, Inc.               Kevin at FutureQuest.net
> (work)
> >              Orlando, Florida kmk at sanitarium.net
> >     <mailto:kmk at sanitarium.net> (personal)
> >              Web page: https://sanitarium.net/ <https://sanitarium.net/>
> >              PGP public key available on web site.
> >
>  ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
> >
> >     --
> >     Please use reply-all for most replies to avoid omitting the mailing
> >     list.
> >     To unsubscribe or change options:
> >     https://lists.samba.org/mailman/listinfo/rsync
> >     <https://lists.samba.org/mailman/listinfo/rsync>
> >     Before posting, read:
> >     http://www.catb.org/~esr/faqs/smart-questions.html
> >     <http://www.catb.org/~esr/faqs/smart-questions.html>
> >
>
> --
> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
>         Kevin Korb                      Phone:    (407) 252-6853
>         Systems Administrator           Internet:
>         FutureQuest, Inc.               Kevin at FutureQuest.net  (work)
>         Orlando, Florida                kmk at sanitarium.net (personal)
>         Web page:                       https://sanitarium.net/
>         PGP public key available on web site.
> ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20211128/0807702c/attachment.htm>