[clug] aggregating disks across multiple machines

Michael James michael at james.st
Sun Oct 25 23:51:18 MDT 2009


I've been asked to recommend a setup for a group of high end  
workstations.
They each have dual 4 core processors, 32 Gig of ram and 2 x 1.5 TB  
disks.
Nice.

At present each machine has a separate 1.5TB /home and /data partition.
Bad:
	no redundancy, data will be lost from a single disk failure.
	files will be copied around as jobs dictate => confusion and waste.
	when partitions start to fill up, files will get put where they fit
		not where they should go => even greater confusion.
	
A much better solution would be aggregate disks
  into a single, large, duplicated, data warehouse
  and have it accessible to all machines.

In the past I'd have moved all the disks to 2 machines,*
  raided them into 2 disk packs,  a master and backup,
  NFS mounted the master on all machines,
  and set up a nightly rsync to refresh the backup.

Nowadays would it be better to use Lustre?

Or is there an updated distributed NFS?
One that can maintain multiple copies of an NFS data repository
  and cache a file locally when needed,
  reflecting changes back the master when necessary.

Or should I look at a global file system?

Which one?

TIA,
michaelj

-- 
Well theme my emoticons disgusted. What has Linux come to?
Michael James

* Yes, I'd find or buy some small disks for the OS on the stripped  
machines.


More information about the linux mailing list