[clug] Distributing a large file to many servers

[Neil Symons]

Another alternative (works well with many files at once)

What about rsync over ssh?

Machine A: 60GB file
Machine B:
Machine C:

If you have 3 machines for example across different networks (or same), run
this in the middle

#On Machine B
rsync -avug -e ssh  <user>@<machineA>:<path> <file>
rsync -avug -e ssh <file> <user<@<MachineC>:Path

This is useful if interruption of tranfer occurs, thus you can resume where
you left off.

-- Neil



On 3/6/06, Simon Burton <simon at arrowtheory.com> wrote:
>
> On Mon, 06 Mar 2006 11:40:34 +1100
> Matthew Sheppard <stray at mac.com> wrote:
>
> >
> > I'm hoping, however, this is one of those problems which was solved
> > neatly many years ago and is simple if you know the right tools.
>
> This may be too heavy duty, but you could use MPI (MPI_Bcast):
> http://www-unix.mcs.anl.gov/mpi/mpich/
>
> either from c code directly or from a python script:
> http://sourceforge.net/projects/mpi4py
>
> (i'm assuming these are 64 bit servers you have).
>
> Simon.
>
> --
> Simon Burton, B.Sc.
> Licensed PO Box 8066
> ANU Canberra 2601
> Australia
> Ph. 61 02 6249 6940
> http://arrowtheory.com
> --
> linux mailing list
> linux at lists.samba.org
> https://lists.samba.org/mailman/listinfo/linux
>