performance problem of using parallel rsync to stage data from 1 source to multiple destination

[Xuehai Zhang]

Paul,

Thanks for your response.

> > results. Why the time of transferring the file to 2N nodes is shorter than
> > twice of the time of transferring the same file to N nodes? Does it make
>
> If the network is not the bottleneck, then cpu or the disks are.

The network is 100Mbps LAN.

> If (similar) tasks are run in parallel, then the data of the files being
> handled may still be in the buffer cache so that it doesn't need to get
> read in from disk again. This will save time...

I agree with you that caching effect might be the cause. The buffer cache
you mentioned refers to the cache on the data source, right?


Xuehai