Is there a better way to transfer data that doesn't use so much cache?

[Rob Campbell]

I've decided to rewrite the script and use cp and mv rather than rsync.  In
the past, I've had some lost data using just cp and mv which is why I moved
to rsync to put the data into a staging directory.  Now that I've been
creating more data (newer cameras with higher megapixel files and more
files), rsync doesn't work as well as it used to.  Trying to get nocache or
something similar to work seemed like it would take more time than to
rewrite the script.

Thanks all for your assistance and suggestions.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In all things, Be Intentional.


On Fri, Aug 5, 2022 at 1:22 AM Wayne Davison via rsync <
rsync at lists.samba.org> wrote:

> On Wed, Aug 3, 2022 at 7:10 PM Dan Stromberg wrote:
>
>> However, if you transfer a large amount of data and do not intend to
>> retransmit that data any time soon, then the memory isn't really put to
>> good use, and can actually cause your system to slow down significantly -
>> particularly if there's a lot of such data transferred.
>>
>
> I have always rejected overcomplicating rsync with cache control code (the
> complexity of a --drop-cache patch I saw was quite horrifying).  In the
> past I pointed people towards https://github.com/Feh/nocache as one way
> to get posix_fadvise used by an rsync copy.  That project now apparently
> suggests creating a memory-bounded cgroup, which sounds interesting.
>
> ..wayne..
> --
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options:
> https://lists.samba.org/mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/rsync/attachments/20220808/48d76e9a/attachment.htm>