[clug] Distributing a large file to many servers
stray at mac.com
Mon Mar 6 00:40:34 GMT 2006
Something which seems like it should be a solved problem, but on which I
haven't been able to dig up a definitive answer.
Given a number of servers, connected through a switch, where one of the
servers has a large (say, 60GB) file, what's the best way to get the
file onto all servers. Obviously having each server individually fetch
the file from the master will be slower than necessary but managing any
other arrangement seems like it will rapidly become quite complex.
Multicast sounds like it would be helpful, but I'm not aware of any
tools for using it in this sort of operation. Bittorrent also seems like
it might be helpful, so if there are any pointers for setting up a
private bittorrent network with CLI only tools that would be great.
I'm hoping, however, this is one of those problems which was solved
neatly many years ago and is simple if you know the right tools.
More information about the linux