OpenLDAP backend broken by the new ldb.tdb event context work, would appreciate advice on how to fix

[Nadezhda Ivanova]

Hi Andrew,
Thank you for your help! It seems, however, that it's not easily 
possible to skip nested contexts - at least not without changes to
source4/libcli/ldap/ldap_client.c, as the client library requires an 
event context.

This context is created when the ldap connection to the backend is 
initialized in lldb_connect. I don't think we can use something other 
than a new context there, else the connection will fail.

I was able to provision successfully by replacing the ldap connection 
context with the one from req->handle when executing each operation 
(lib/ldb-samba/ldb_ildap.c:405 in the attached patch).
This works for provisioning, but is decidedly not a permanent fix - its 
just there to prove we are on the right track.
A much better way would be to modify the ldap client library to 
optionally use the per request context, if provided.

Best Regards,
Nadya


On 07/20/2017 03:40 AM, Andrew Bartlett wrote:
> On Wed, 2017-07-19 at 17:00 +0300, Nadezhda Ivanova wrote:
>> Hi team and Andrew,
>> I'm trying to start using the newest Samba for my OpenLDAp backend work
>> - after the long hiatus, I was using code that was 2 years old, so I
>> decided to adapt to the newest master.
> 
> It is so good to see you back at this.  It has been a long time!
> 
>> However, we are no longer able to provision an OpenLDAP backend - one of
>> the issues I have a workaround for, by adding "client ldap sasl wrapping
>> = plain" to my smb.conf, until I'm able to fix it properly, but the
>> other I need some advice on.
>>
>> While trying to create the domain naming context, provisioning hangs here:
>>
>>
>> #0  0x00007ffff78f1673 in __epoll_wait_nocancel () at
>> ../sysdeps/unix/syscall-template.S:81
>> #1  0x00007ffff5b0a074 in epoll_event_loop (epoll_ev=0x1ec7360,
>> tvalp=0x7fffffffc640) at ../lib/tevent/tevent_epoll.c:642
>> #2  0x00007ffff5b0a98c in epoll_event_loop_once (ev=0x1ec70e0,
>> location=0x7ffff64b0edd "../lib/ldb/common/ldb.c:637") at
>> ../lib/tevent/tevent_epoll.c:930
>> #3  0x00007ffff5b07693 in std_event_loop_once (ev=0x1ec70e0,
>> location=0x7ffff64b0edd "../lib/ldb/common/ldb.c:637") at
>> ../lib/tevent/tevent_standard.c:114
>> #4  0x00007ffff5b0050f in _tevent_loop_once (ev=0x1ec70e0,
>> location=0x7ffff64b0edd "../lib/ldb/common/ldb.c:637") at
>> ../lib/tevent/tevent.c:721
>> #5  0x00007ffff64a71bc in ldb_wait (handle=0x15a1fb0, type=LDB_WAIT_ALL)
>> at ../lib/ldb/common/ldb.c:637
>> #6  0x00007fffda0d3d7e in new_partition_set_replicated_metadata
>> (ldb=0x1c52460, module=0x1b77090, last_req=0x1d5a070, data=0x15f8bd0,
>> partition=0x1c75bd0)
>>       at ../source4/dsdb/samdb/ldb_modules/partition_init.c:601
>> #7  0x00007fffda0d46da in partition_create (module=0x1b77090,
>> req=0x174c980) at ../source4/dsdb/samdb/ldb_modules/partition_init.c:827
>> #8  0x00007fffda0d21e2 in partition_extended (module=0x1b77090,
>> req=0x174c980) at ../source4/dsdb/samdb/ldb_modules/partition.c:1172
>> #9  0x00007ffff6488c42 in ldb_next_request (module=0x1b77090,
>> request=0x174c980) at ../lib/ldb/common/ldb_modules.c:543
>> #10 0x00007fffdaefe427 in np_part_search_callback (req=0x2162590,
>> ares=0x15a47a0) at ../source4/dsdb/samdb/ldb_modules/new_partition.c:143
>> #11 0x00007fffd6fd7275 in ltdb_request_done (ctx=0x1c9b4c0, error=32) at
>> ../lib/ldb/ldb_tdb/ldb_tdb.c:1314
>> #12 0x00007fffd6fd7556 in ltdb_callback (ev=0x1ec70e0, te=0x2303ba0,
>> t=..., private_data=0x1c9b4c0) at ../lib/ldb/ldb_tdb/ldb_tdb.c:1424
>> #13 0x00007ffff5b08602 in tevent_common_loop_timer_delay (ev=0x1ec70e0)
>> at ../lib/tevent/tevent_timed.c:369
>> #14 0x00007ffff5b0a90c in epoll_event_loop_once (ev=0x1ec70e0,
>> location=0x7ffff64b0edd "../lib/ldb/common/ldb.c:637") at
>> ../lib/tevent/tevent_epoll.c:915
>> #15 0x00007ffff5b07693 in std_event_loop_once (ev=0x1ec70e0,
>> location=0x7ffff64b0edd "../lib/ldb/common/ldb.c:637") at
>> ../lib/tevent/tevent_standard.c:114
>> #16 0x00007ffff5b0050f in _tevent_loop_once (ev=0x1ec70e0,
>> location=0x7ffff64b0edd "../lib/ldb/common/ldb.c:637") at
>> ../lib/tevent/tevent.c:721
>> #17 0x00007ffff64a71bc in ldb_wait (handle=0x15af320, type=LDB_WAIT_ALL)
>> at ../lib/ldb/common/ldb.c:637
>> #18 0x00007ffff68c4af2 in py_ldb_add (self=0x7fffde1f1eb0,
>> args=0x7fffd647d908, kwargs=0x0) at ../lib/ldb/pyldb.c:1415
>> #19 0x0000000000523f6d in PyEval_EvalFrameEx ()
>> #20 0x0000000000555551 in PyEval_EvalCodeEx ()
>>
>>
>> This seems to start happening after this commit:
>>
>> commit e67d3568e5853387d0675ed13d51df21e6274770
>> Author: Andrew Bartlett <abartlet at samba.org>
>> Date:   Fri May 12 02:30:01 2017 +0200
>>
>>       ldb: Use the private event context in ldb_tdb and ldb_wait()
>>
>>       This enables the previous commits, and ensures that ldb_tdb is safe
>> from operations while locks
>>       are held
>>
>>       Signed-off-by: Andrew Bartlett <abartlet at samba.org>
>>       Reviewed-by: Stefan Metzmacher <metze at samba.org>
>>       Reviewed-by: Garming Sam <garming at catalyst.net.nz>
>>
>>
>>
>> Could you advise me on why this happens, and possible steps to fix?
> 
> ldb_ildap.c:ildb_connect()
> 	ildb->event_ctx = ldb_get_event_context(ldb);
> 
> This uses one of those evil nested event loops, which are essentially
> banned in Samba and now also ldb, for causing too much mischief.
> 
> Instead, we now create a new event context for every request, so for
> every request, you need to find the per-request event context and wait
> on that instead.
> 
> Eliminating long-term variables with the event context in it, and
> instead using ldb_handle_get_event_context(req->handle) would make a
> big difference.
> 
> Andrew Bartlett
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0004-s4-openldap-Remove-nested-event-contexts-from-ildap-.patch
Type: text/x-patch
Size: 3436 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20170727/8054b324/0004-s4-openldap-Remove-nested-event-contexts-from-ildap-.bin>