[ceph-users] CTDB Cluster Samba on Cephfs

[Matthieu Patou]

On 03/28/2013 07:41 AM, Sage Weil wrote:
> On Wed, 27 Mar 2013, Matthieu Patou wrote:
>> On 03/27/2013 10:41 AM, Marco Aroldi wrote:
>>> Hi list,
>>> I'm trying to create a active/active Samba cluster on top of Cephfs
>>> I would ask if Ceph fully supports CTDB at this time.
>> If I'm not wrong Ceph (even CephFS) do not support exporting a block device or
>> mounting the same FS more than once whereas CTDB explicitly require that you
>> have a distributed filesystem where the same filesystem is mounted across all
>> the nodes.
> Er, sort of.
>
> RBD presents a block-based interface.  As many clients as you want can use
> that at the same time, although the client caching should be disabled if
> the users don't call flush() to make their writes visible to others.
> Beyond that, RBD itself doesn't care how many people use it.  *However*,
> if you put a "local file system" on top of RBD (like ext4, xfs, btrfs,
> zfs, ...), only one client should use RBD at a time because those file
> systems are designed for exclusive access to the disk.  If you use a
> "clustered file system" like ocfs2 or gfs[2], multiple clients can share
> the same RBD volume in a useful way, but they expect coherency to behave
> like on a SAN (which means writes are visible immediately but not durable
> until flush), which means RBD caching should be turned off.
>
> CephFS is designed for concurrent, shared, coherent access from many many
> clients.  Think NFS, but scalable, and coherent/consistent when clients
> are accessing the same files and directories.
Ok my bad, I misunderstood, the documentation of Cephfs was not super 
clear on this point I assumed that if it wasn't stated then it means 
that multiple coherent client access wasn't a feature.


Matthieu.