[ceph-users] CTDB Cluster Samba on Cephfs

[Simo]

On 04/03/2013 07:24 PM, Jeremy Allison wrote:
> On Wed, Apr 03, 2013 at 03:51:45PM -0700, Jeremy Allison wrote:
>> On Wed, Apr 03, 2013 at 06:26:05PM -0400, Simo wrote:
>>> On 04/03/2013 05:42 PM, Jeremy Allison wrote:
>>>> On Wed, Apr 03, 2013 at 03:53:58PM -0500, Sam Lang wrote:
>>>>> On Thu, Mar 28, 2013 at 6:32 AM, Kai Blin <kai at samba.org> wrote:
>>>>>> -----BEGIN PGP SIGNED MESSAGE-----
>>>>>> Hash: SHA1
>>>>>>
>>>>>> On 2013-03-28 09:16, Volker Lendecke wrote:
>>>>>>> On Wed, Mar 27, 2013 at 10:43:36PM -0700, Matthieu Patou wrote:
>>>>>>>> On 03/27/2013 10:41 AM, Marco Aroldi wrote:
>>>>>>>>> Hi list, I'm trying to create a active/active Samba cluster on
>>>>>>>>> top of Cephfs I would ask if Ceph fully supports CTDB at this
>>>>>>>>> time.
>>>>>>>> If I'm not wrong Ceph (even CephFS) do not support exporting a
>>>>>>>> block device or mounting the same FS more than once whereas CTDB
>>>>>>>> explicitly require that you have a distributed filesystem where
>>>>>>>> the same filesystem is mounted across all the nodes.
>>>>>>> Is that true? I thought Ceph was one of the cluster filesystems
>>>>>>> doing just that. What is Ceph if not a cluster file system?
>>>>>> There's some problem with mounting the in-kernel cephfs driver on
>>>>>> systems running the osd, iirc. I had to use the fuse-based driver to
>>>>>> mount, which obviously is not too great, speed-wise.
>>>>>> See http://ceph.com/docs/master/faq/#try-ceph for a better description
>>>>>> of the issue.
>>>>> Just to let folks know, we have a ceph vfs driver for samba that we
>>>>> are testing out now.  We're planning to resolve a few of the bugs that
>>>>> we're seeing presently with smbtorture, and send a pull request to the
>>>>> samba repo.  If anyone wants to help with testing, let us know.  The
>>>>> changes currently reside in the ceph branch of
>>>>> http://github.com/ceph/samba.
>>>> One thing I noticed. Returning a struct ceph_dir_result *
>>>> cast to a DIR * struct won't work in 4.0.x as we expect
>>>> dirfd(DIR *) to return a valid file descriptor.
>>> This sounds like a bug in 4.0.x, we should add a VFS layer operation
>>> for dirfd if we do not have it already.
>> Yeah, I'd started thinking about that.
>>
>> Problem is that DIR * is an opaque structure
>> pointer, and systems aren't guarenteed to have
>> dirfd() at all (we have to test for it).
>>
>> Hmmm. I guess we can get away with having
>> SMB_VFS_DIRFD() returning -1, ENOSYS in
>> the unsupported system case and then
>> just moving the fallback code out of the #ifdef.
> So I can make this fix for 4.1.x, but not in 4.0.x
> obviously, due to the VFS-freeze policy.
>
> I'll take a look..

Should we forcibly compile out dirfd() usage until 4.1.x then ?

Simo.