Samba cluster support codebase

[James Peach]

On Tue, 2006-03-21 at 18:13 +0100, Volker Lendecke wrote:
> On Tue, Mar 21, 2006 at 10:47:33AM -0600, Abhijith Das wrote:
> > I was told (in a discussion on the samba mailing list: 
> > http://lists.samba.org/archive/samba/2006-March/119252.html ) that 
> > clustering support for Samba3 is being developed and some code/patches 
> > exist. I was hoping to get my hands on this code to play around with 
> > some cluster-specific aspects of Samba. Can somebody point me to the 
> > right code/patches/CVS-tree where this cluster-specific code is being 
> > worked upon?
> 
> There's two branches where work has been done:
> 
> branches/tmp/jpeach-cluster is work done by James Peach from
> SGI. It is based on 3.0.14a, parts of it are pretty
> CXFS-specific.

The CXFS-specific parts are hidden behind an (relatively) abstract
interface. It should be quite easy to add support for different types of
cluster.

> Independently of that (I did not know he was working on it)
> I started branches/tmp/vl-cluster. This is not finished
> work, it contains some infrastructure work to get cluster
> messaging done and have the locking.tdb api based on
> messages.
> 
> It also contains some work to use files per key for
> locking.tdb, which on some cluster file systems might be
> faster than beating a shared file with little reads and
> writes.
> 
> I mentioned that James' work is based on 3.0.14a because
> while I started to look at clustering I figured that the
> then-current share mode and oplock implementation makes
> clustering not particularly easy. Not that it's easy now,
> but Jeremy rewrote our share mode code and I did the same
> for the oplock stuff.
> 
> > Also, documentation regarding what works and what needs to be 
> 
> I don't know about James' branch, "mine" does not work yet,
> I was interrupted with other stuff.

Mine works. However, there will be certain workloads where it won't
perform as well as the standalone case.

> > implemented would be very helpful.
> 
> The main problem that we have is speed. Both on the pure
> Samba side as well as in the cluster file system side. I'm
> not too familiar with GFS, but I would suspect that its
> strengths are in handling large files with sequential
> read/write operations. Windows clients do things to the file
> system that are completely contrary to that: Many
> opens/closes in sequence, weird read/write patterns, wild
> locking and so on. You might want to look at the routine
> open_file_ntcreate in smbd/open.c. If we get that _FAST_
> cluster-wide, I think most of our problems are solved.

I generally agree with this. Given the different performance
characteristics of different clusters I think we have to accept that
there will always be some workloads that don't perform well for
particular cluster implementations. Our challenge is to make sure that
the Samba implementation is never the limiting factor.

> Byte range locks are another topic, but I think (did not measure
> it) would not have a dramatic effect on the overall cluster
> performance if the locking daemons for brlocks are
> well-spread.

This is an unresolved architectural issue :)

-- 
James Peach | jpeach at samba.org