[clug] Distributing a large file to many servers
neil.symons at gmail.com
Mon Mar 6 01:53:22 GMT 2006
Another alternative (works well with many files at once)
What about rsync over ssh?
Machine A: 60GB file
If you have 3 machines for example across different networks (or same), run
this in the middle
#On Machine B
rsync -avug -e ssh <user>@<machineA>:<path> <file>
rsync -avug -e ssh <file> <user<@<MachineC>:Path
This is useful if interruption of tranfer occurs, thus you can resume where
you left off.
On 3/6/06, Simon Burton <simon at arrowtheory.com> wrote:
> On Mon, 06 Mar 2006 11:40:34 +1100
> Matthew Sheppard <stray at mac.com> wrote:
> > I'm hoping, however, this is one of those problems which was solved
> > neatly many years ago and is simple if you know the right tools.
> This may be too heavy duty, but you could use MPI (MPI_Bcast):
> either from c code directly or from a python script:
> (i'm assuming these are 64 bit servers you have).
> Simon Burton, B.Sc.
> Licensed PO Box 8066
> ANU Canberra 2601
> Ph. 61 02 6249 6940
> linux mailing list
> linux at lists.samba.org
More information about the linux