Max number of files to transfer
cooke at sun.ac.za
Fri Jul 7 08:17:51 GMT 2006
Wayne Davison wrote:
> On Thu, Jul 06, 2006 at 01:59:54PM -0400, Bob Bagwill wrote:
>> You probably already considered this, but you could also do something
>> like this: [...]
> That divides the entire file-list up into 50-file chunks, whereas I
> believe that the original poster wanted to allow 50 differing files to
> be updated before quitting (out of 50+N files checked for being
Yes this is correct. The link that is being used is the limiting factor
and the time that the link is available is also a limiting factor. So I
want to be able to quickly startup rsync(by limiting the file scanning)
and then transfer those in one go, and not create and terminate rsync
sessions as I go.
> If the file-system scan is not the limiting factor that is slowing
> things down, then running rsync with --dry-run could be used to output a
> list of the files that need to be updated, and then that list could be
> divided into groups of 50 and iterated over.
I dont think this wont really work because it will take to long to run
because the link is to slow. This is because with --dry-run rsync still
has to talk to the remote host if I understand it correctly and I do not
For the remote to local transfer the problem should be solved by using a
command to be passed to the remote shell as described in
http://rsync.samba.org/examples.html "Fancy footwork with remote file
lists". For the local to remote transfer I will take a look at the src
or will just keep two directories and transfer the correct amount of
files into the directory that I am going to rsync before sending.
Once the files are placed in the directory they will not change. In a
previous post Matt McCutchen said that I can use --append to improve
this start up time.
In the man page it says that this assumes that if the destination file
is shorter than the source file it will then assume that the rest of the
file must simply be appended to the file in the destination. Will this
always be the case here because the source file never changes? It says
that it implies --inplace but I need to used --partial-dir, will this
cause a conflict?
More information about the rsync