[clug] Distributing a large file to many servers

[Matthew Sheppard]

Hi all,

Something which seems like it should be a solved problem, but on which I 
haven't been able to dig up a definitive answer.

Given a number of servers, connected through a switch, where one of the 
servers has a large (say, 60GB) file, what's the best way to get the 
file onto all servers. Obviously having each server individually fetch 
the file from the master will be slower than necessary but managing any 
other arrangement seems like it will rapidly become quite complex.

Multicast sounds like it would be helpful, but I'm not aware of any 
tools for using it in this sort of operation. Bittorrent also seems like 
it might be helpful, so if there are any pointers for setting up a 
private bittorrent network with CLI only tools that would be great.

I'm hoping, however, this is one of those problems which was solved 
neatly many years ago and is simple if you know the right tools.

Thanks,
Matt Sheppard