CTDB

[Andrew Bartlett]

On Thu, 2007-08-16 at 15:19 -0600, Andrew B. Lundgren wrote:
> I am evaluating setting up a several terabyte cluster on Linux.  I am 
> currently working to select the cluster software to use and have a few 
> questions about CTDB and samba.
>  
> I guess the first one should be how stable is CTDB?  Would you consider 
> it production ready?

It is certainly more production ready than the alternatives for Samba on
clustered filesystems (which have too often boiled down to 'pray
hard' :-)

> Given a 4 node cluster, with file A residing on node 2.  When a windows 
> client requests file A, how does the data get to it?

The design CTDB is for doesn't have the file residing on node2 as much
as residing on a SAN, with access via any node.  If your 'SAN' actually
happens to be a cluster of servers with local disk, then perhaps you are
really looking for an MSDFS-based redirect setup (where your virtual
cluster simply redirects clients to the correct node to find their
file). 

> Will the client go to any random node in the cluster and request the 
> file A and then be directed to node 2 to pull the file, or will the file 
> be pulled to the random node from node 2 and then sent to the client?  

To avoid that double-hop, then you probably want the MSDFS design.  

If your kernel-level cluster FS hides the detail about where the file
actually is, then CTDB doesn't care about the details, as long as open()
gets the file. 

> I have not settled on the clustering technology to use.  I think I have 
> narrowed a little bit down to lustre, gluster or RHES/CentOS 5 cluster 
> file system.
>  
> I like the fact that RH includes the cluster with the distro.  
> I like that from what I can tell, gluster does not require a custom kernel.

GFS is in the upstream kernel, but gluster is a FUSE module.  My gut
feeling is that I don't trust the FUSE concept quite yet...

It would be very interesting to know how gluster supports the locking
required without inter-node communication. 

> I like that lustre is one of the primary test beds for CTDB.

I would have said that IBM GPFS was the primary test bed for CTDB.  Any
complying FS (see the ping-pong test) will work, as CTDB was explicitly
designed to be FS-agnostic. 

> Are there any stark differences with CTDB and these clusters that should 
> drive a preference of one of them?
 
CTDB shouldn't care, so the main task is to look at any features you
particularly need.

Andrew Bartlett

-- 
Andrew Bartlett
http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org
Samba Developer, Red Hat Inc.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.samba.org/archive/samba-technical/attachments/20070817/8f8e0b54/attachment.bin