[ROADMAP] Catalyst's focus on Samba

[Garming Sam]

On 12/03/17 21:51, Andrew Bartlett via samba-technical wrote:
> Scale
> -----
>
> Tied up in performance work is a desire to have samba scale more, to
> serve more user and hold more groups/group members.  The direction of
> this work is less certain, but having broken the back of the 'too many
> links melts Samba' issue, I'm sure we will be asked to do more here.  
>
> Specific possibilities include LMDB and a GUID-based index scheme.
>
I spent some time looking at abstracting ldb_tdb to interface with a generic key-value store. This was so that LMDB could be plugged in as minimally as possible. There is a quick lmdb backend hacked together partly from Jakub's patches and it's still a mess of patches (and I hadn't quite resolved the build system issues with where to build the library), but the core functionality was there. It could provision and run Samba tests normally and used all the existing indexing schemes.

The interfaces currently use TDB_DATA which should be converted to DATA_BLOB instead. 

With some synthetic tests, it improved write performance noticeably despite there being no large structural changes. However under some of the test loads (virtual list view tests), the overall performance was noticeably slower which needs to be investigated. However, if the goal is to bypass the 32-bit limit, then the current arrangement should suffice.

Of the key-value interface operations to implement, I believe the case required for re-indexing was still incomplete.

There's also a bug I found in regards to read-lock counts being broken in ldb_tdb, but I haven't established the consequences of such a bug.


http://git.catalyst.net.nz/gitweb?p=samba.git;a=shortlog;h=refs/heads/lmdb-wip2


Cheers,

Garming