Passing result of shuf to rsync

[hancooper]

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Saturday, September 18, 2021 12:58 AM, Kevin Korb <kmk at sanitarium.net> wrote:

> Quoting from the man page....
>
> SORTED TRANSFER ORDER
> Rsync always sorts the specified filenames into its internal
> transfer list. This handles the merging together of the contents of identi‐
> cally named directories, makes it easy to remove duplicate
> filenames, and may confuse someone when the files are transferred in a
> differ‐
> ent order than what was given on the command-line.

So what I am doing is simply rubbish.  I need to make separate rsync calls.  If I try to put the results in an array, in there a limit to the number of files that can be processed?



> If you need a particular file to be transferred prior to another,
> either separate the files into different rsync calls, or consider using
> --delay-updates (which doesn't affect the sorted transfer order,
> but does make the final file-updating phase happen much more rapidly).
>
> and...
>
> NOTE: sorting the list of files in the --files-from input
> helps rsync to be more efficient, as it will avoid re-visiting the path
> elements that are shared between adjacent entries. If
> the input is not sorted, some path elements (implied directories) may end
> up being scanned multiple times, and rsync will eventually
> unduplicate them after they get turned into file-list elements.
>
> On 9/17/21 8:41 PM, hancooper wrote:
>
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Saturday, September 18, 2021 12:29 AM, Kevin Korb kmk at sanitarium.net wrote:
> >
> > > Well, what you have should function. But rsync is going to sort the list.
> >
> > I want rsync to use the list as given to it by shuf.
> >
> > > On 9/17/21 8:26 PM, hancooper wrote:
> > >
> > > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > > On Saturday, September 18, 2021 12:06 AM, Kevin Korb via rsync rsync at lists.samba.org wrote:
> > > >
> > > > > Not really sure what you are trying to accomplish here. Seems like it
> > > > > should work the way you have it. Note that many wonky rsync kludges are
> > > > > due to people not realizing that rsync can have multiple source
> > > > > arguments. Instead of the source simply being . it can be a list of
> > > > > stuff. Also, note that -a includes --recursive except when --files-from
> > > > > is in play.
> > > >
> > > > I am trying send via rsync a shuffled list of files.
> > > >
> > > > > On 9/17/21 7:11 PM, hancooper via rsync wrote:
> > > > >
> > > > > > I am trying to pass filename results that have been shuffled, then pass
> > > > > > the file to rsync.
> > > > > > Not quite sure whether to use --files-from=- or some other way.
> > > > > > shuf -n "$nf" -e ${dpath}/${incl} |
> > > > > >     rsync -av --update --files-from=- . "$dst"
> > > > >
> > > > > --
> > > > > ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,
> > > > > Kevin Korb Phone: (407) 252-6853
> > > > > Systems Administrator Internet:
> > > > > FutureQuest, Inc. Kevin at FutureQuest.net (work)
> > > > > Orlando, Florida kmk at sanitarium.net (personal)
> > > > > Web page: https://sanitarium.net/
> > > > > PGP public key available on web site.
> > > > >
> > > > > ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,
> > > > >
> > > > > -----------------------------------------------------------
> > > > >
> > > > > Please use reply-all for most replies to avoid omitting the mailing list.
> > > > > To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync
> > > > > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
> > >
> > > --
> > > ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,
> > >
> > > Kevin Korb Phone: (407) 252-6853
> > >
> > > Systems Administrator Internet:
> > >
> > > FutureQuest, Inc. Kevin at FutureQuest.net (work)
> > >
> > > Orlando, Florida kmk at sanitarium.net (personal)
> > >
> > > Web page: https://sanitarium.net/
> > >
> > > PGP public key available on web site.
> > >
> > > ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,
>
> --
>
> ~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,-~'`^`'~-,._.,
> Kevin Korb Phone: (407) 252-6853
> Systems Administrator Internet:
> FutureQuest, Inc. Kevin at FutureQuest.net (work)
> Orlando, Florida kmk at sanitarium.net (personal)
> Web page: https://sanitarium.net/
> PGP public key available on web site.
> ~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,..,-~'`^`'~-,._.,