[clug] Need read-only NFS with local disk based caching

[Michael.James at csiro.au]

Pardon me asking this not-particularily Reiser related question,
 but I need to tap some file-system expertise.

On a cluster of 66 nodes,  (suse 8.2, reiser 3.6)
 our bioinformatic database files are outgrowing
 the nodes capacity to store them,  (>24Gig).

We call them databases, but to an operating system
 they are just a bunch of big (~2Gig) files
 that need to be read at the beginning of each job.

Plenty of space on the fileserver but we find
 when the databases are provided through NFS
 that cluster throughput levels off at about 10 nodes.
Seems the single NFS server becomes a bottleneck.
It might be possible to tune this to do better
 but it would take a lot of improvement
 to get all 66 nodes humming.

We need something that provides a network file system
 but with local caching.  Sun's cachefs would be ideal.
Note that this is the simplest case, read-only access,
 infrequent (weekly at most) changes.
Manually flushing out-of-date copies would be acceptable.

After such a change or when someone asks for something new
 we have a short pause while the information is re-distributed
 and we are humming again.

I thought to use openAFS or CODA
 but find a wheird mix of doco dating from 1900s.

AFS is proud of being able to handle 8 Gig volumes!
Bioinformatic software tries (unsucessfully)
 to keep filesizes below 2 Gig for historical reasons.
So if each file is a seperate volume I _could_ live with an 8 gig limit.

Coda suffers the whierdness of its AFS roots
 and I can't find doco on what its limitations are!

I haven't had much luck posting this question
 in my usual haunts
 so Thanks for any suggestions,

michaelj

-- 
Michael James				michael.james at csiro.au
System Administrator			voice:	02 6246 5040
CSIRO Bioinformatics Facility	fax:		02 6246 5166