[PATCH] Core ldb modules changes

[Andrew Bartlett]

On Thu, 2008-01-17 at 18:48 -0500, simo wrote:
> On Fri, 2008-01-18 at 10:41 +1100, Andrew Bartlett wrote:
> > On Thu, 2008-01-17 at 18:26 -0500, simo wrote:
> > > On Fri, 2008-01-18 at 10:25 +1100, Andrew Bartlett wrote:
> > > > On Thu, 2008-01-17 at 10:37 -0500, simo wrote:
> > > > > This is a new patch[1] to the core ldb internal async API to address some
> > > > > issue that were preventing me from using ldb in a full async server.
> > > > > The next step will be to add support to replace the run queue now
> > > > > attached to the ldb context with an event system and completely stop
> > > > > using the ldb_wait() call in fully async code calling code.
> > > > 
> > > > > Before committing I will finish porting the dsdb modules to this
> > > > > interface, but I thought that people may be interested in commenting and
> > > > > understanding the direction I want to take with this code.
> > > > > 
> > > > > Simo.
> > > > > 
> > > > > [1] Patch here as it is a bit too big for the list:
> > > > > http://samba.org/~idra/patches/0001-This-commit-changes-in-some-key-areas-the-way-ldb-mo.patch
> > > > > 
> > > > > Also git tree here: git://git.samba.org/idra/samba.git
> > > > > branch v4-0-ldb-async
> > > > 
> > > > - search request callbacks MUST not issue new requests until the
> > > >   LDB_REPLY_DONE ldb_reply is received. Once this reply is received a
> > > >   module can issue a new request or return to the upper module using
> > > >   ldb_module_done()
> > > > 
> > > > 
> > > > Why do you want this restriction?  In a number of modules, this pattern
> > > > has allowed an easy, asynchronous handling of some difficult situations
> > > > (such as handling subtree renames, without building massive lists in
> > > > memory). 
> > > 
> > > Which only worked by chance
> > 
> > It seemed to work perfectly fine in ldb_tdb and against LDAP...
> 
> Yes probably of a side effect of calling ldb_wait and intercepting it it
> was working even with LDAP, but trust me it was *really* dirty.

I still think this is a regression, and aside from your feeling that it
was 'dirty', I still don't see why it had to be lost.

> > >  because ldb_tdb ws async and giving back
> > > results before the former request was done (this does not happen anymore
> > > now even with ldb_tdb).
> > > 
> > > It simply does not work.
> > 
> > Why not?  I'm confused - it seems a very natural pattern to 'do
> > something' in the callback, where that something may well be a
> > modification to the records.
> > 
> > Why should this not be possible?  How do you suggest implementing
> > 'iterate and fix' modules?
> 
> See ldb_map I collect the data and the fire a new request when I get
> LDB_REPLY_DONE, repeat until done.

The point of an async API is to avoid having to collect data.  With this
limitation, we may as well make it sync. 

> This also helps getting just one outstanding call at a time, with the
> previous method you could end up with countless outstanding calls and
> some ldap server has limitations on the number of calls each single
> client can have outstanding, so the previous way may also lead to
> unpredictable results.

I'm prepared to bear with that risk, or control that when issuing new
requests.  I think being unable to usefully use the callback provided is
an important regression. 

Andrew Bartlett

-- 
Andrew Bartlett
http://samba.org/~abartlet/
Authentication Developer, Samba Team           http://samba.org
Samba Developer, Red Hat Inc.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://lists.samba.org/archive/samba-technical/attachments/20080118/e70862d2/attachment.bin