s4 ldb tdb limits

[Oliver Liebel]

Am 27.08.2015 um 14:31 schrieb Nadezhda Ivanova:
> Hi Oliver,
> I don't think I can be particularly helpful in this matter - the scope 
> and goal of our project is quite different - we seek to replace the 
> Samba LDAP service with OpenLDAP, rather than just write a new ldb 
> backend. If Jakub or you have any specific questions, I'll see if I 
> can be of assistance.
>
> Best Regards,
> Nadya

Hi Nadya,
yes, i know what the goals are, and that they are of course different.
It was just ment exactly for that case: assistance in -maybe a few- 
specific questions,
if necessary, and possible.

Thanks for offering your assistance.

Thanks
Oliver

>
> On 08/27/2015 03:23 PM, Oliver Liebel wrote:
>>
>>
>> Am 27.08.2015 um 00:21 schrieb Oliver Liebel:
>>>
>>> Am 26.08.2015 um 18:02 schrieb Jeremy Allison:
>>>> On Wed, Aug 26, 2015 at 03:46:41PM +0200, Oliver Liebel wrote:
>>>>> Am 26.08.2015 um 04:34 schrieb Andrew Bartlett:
>>>>>> On Tue, 2015-08-25 at 09:34 -0700, Jeremy Allison wrote:
>>>>>>
>>>>>>> Yeah, lmdb and backending with OpenLDAP is our
>>>>>>> long term solution for these limits I think.
>>>>>>>
>>>>>>> Nadia, any progress report on migrating us over ?
>>>>>>>
>>>>>>
>>>>>>
>>>>>> Jeremy,
>>>>>>
>>>>>> I should caution you that while moving to lmdb rather than tdb 
>>>>>> may be
>>>>>> reasonably practical (a prototype has been developed so far), the 
>>>>>> task
>>>>>> of moving to using OpenLDAP is I feel an order of magnitude 
>>>>>> larger than
>>>>>> the task to move us to using MIT Kerberos.  Equally, when 
>>>>>> finished it
>>>>>> promises great improvements, but we should be very clear that it 
>>>>>> needs
>>>>>> commensurate resources.
>>>>>>
>>>>>> The issue is this:
>>>>>>
>>>>>> At it's heart, Samba4 turned out to be a series of RPC services
>>>>>> clustered around the LDB module stack.  The vast majority of the
>>>>>> complexity is in that stack.   As I understand it, the OpenLDAP 
>>>>>> backend
>>>>>> project seeks to 'simply' replace that stack, using a number of 
>>>>>> Samba
>>>>>> libraries in the process.
>>>>>>
>>>>>> I hope this helps give an idea of the scale involved here.
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>> Andrew Bartlett
>>>>> I fully agree with Andrew.
>>>>>
>>>>>  From my point of view (and that of many customers) there are at 
>>>>> this point
>>>>> 2 major tasks to get S4 into larger/enterprise scale:
>>>>>
>>>>> - Fast (LM)DB Backend
>>>>> - W2K12 DC compatibility
>>>>>
>>>>> I know for sure how big the task is, to get OpenLDAP as an
>>>>> S4 Backend with full Schema Semantics and DRSUAPI or any kind of DRS
>>>>> working, as Andrew and I already worked a few years ago on the
>>>>> S4/OpenLDAP Backend.
>>>>>
>>>>> And i would love to see OpenLDAP as an S4 Backend, but from a
>>>>> realistic point of
>>>>> view, this task can't be finished very shortly, because its really 
>>>>> complex.
>>>>>
>>>>>
>>>>> I think in the meantime it is the best approache,
>>>>> to get the LDB/TDB Stack enhanced / replaced with LMDB,
>>>>> so that large(r) scale Installations could be satisfied.
>>>>> We should get ' the best Database under the hood ' working. The
>>>>> quicker, the better.
>>>>>
>>>>> One side effect: When the S4/OpenLDAP Job is done, we got already 
>>>>> a mostly
>>>>> compatible LMDB-Backend inside S4 working.
>>>>>
>>>>>
>>>>> @Andrew, Nadezhda and Jakub:
>>>>> could you please submitt  a full description / explanation of the
>>>>> minor tasks
>>>>> to get TDB replaced with LMDB.
>>>>> I will try to get some hr, to get this done (but no promises,
>>>>> i'll keep you informed)
>>>> Yeah, I wasn't saying it was going to be easy :-).
>>>>
>>>> That's why I ust asked Nadia for a status update,
>>>> nothing more :-).
>>>
>>> Jakub pinged me, in the next days
>>> he'll bring me up to date about the oustanding
>>> tasks regarding TDB / LMDB  Replacement.
>>>
>>>
>>
>> Update:
>>  It looks like i could get shortly 2 Workers/Coders for the TDB <-> 
>> LMDB Replacement Task
>> to support Jakub.
>> @ Nadehzda: Maybe some additional Advices / Hints / Whatever... 
>> regarding LMDB Integration
>> from Symas could be helpful  - if possible.
>>
>> Thanks in Advance
>> Oliver
>>
>>
>>
>>
>>
>