lbd modules and recursion

[Simo Sorce]

On Tue, 2005-12-27 at 20:46 +1100, Andrew Bartlett wrote:
> On Tue, 2005-12-27 at 10:33 +0100, Simo Sorce wrote:
> > On Tue, 2005-12-27 at 20:06 +1100, Andrew Bartlett wrote:
> > 
> > > I'm very worried that in the real world, with potentially arbitrary
> > > modules, that such loops would be unavoidable, if modules were to call
> > > back up the stack.
> > 
> > Only if modules programmers do not make a rule of preventing them.
> 
> Or the infrastructure it setup to be less painful.

Less painful and less functional ?
Yes we need to search for a compromise, but let's try to see all
aspects.

> You make a big point of the schema violation prevention.  However you
> miss my point.  If a module can do 'bad things' attached to the original
> request, which clearly must go to ldb_next_request(), what do you gain
> by saying it cannot issue new requests at that layer?

That said module, at least, do not break the tree.

> My point is that a module should have a consistant world view.  It is
> making modifications to what it can see of the database below it.
> Therefore, all searches/modifications/deletes should be in reference to
> what it can see below it.  What an application may see form above is
> irrelevant.  

It depends on what is "consistent", the primary goal is to achieve some
operation, bu the main restriction is that the tree must remain
consistent to the outside world. The only consistent view that matters
is the view seen at the API level.
Modules generally are so specialized that they care only of a little
portion of the tree or only of some aspects of the tree, not the whole.
 
> > you should use a token system of some sort inside your modules
> > private_data to avoid recursions, but from what you say your
> > modification will not catch your code as you are not touching
> > userPassword yourself, so that's not a problem, the second call will
> > pass through your module and recursion is avoided.
> 
> I think this is creating more work, and I don't see the gain.

Consistency.

> I think it is quite sane to have modules which depend on other modules.
> I also expect we will consolidate many of these modules.  

I agree, we will probably consolidate some of the modules in the dsdb
tree, but this does not mean we must make up an interface that force us
to consolidate module to keep consistency.

> > You MUST be prepared to handle recursion because the module you call
> > with ldb_next_request() may call a new ldb_modify() operation.
> 
> I think this is the part that is creating problems.  If you remove this
> MUST (which does not currently exist in real code), I think we have a
> simpler system.

It does exist, there already are modules that make ldb calls inside
themselves and when they have been made I tried to keep them logically
safe, if they are not then work should be done o make them safer.

> > It may not be a direct call it can be any sequence of calls.
> 
> Indeed!

but i is unlikely that such rare events will cause loops, we do not
introduce modules without thinking about interactions after all.

> > As a general rule calling the stack from the start is the right thing.
> > In some special case we may consider about breaking this rule, but I do
> > not see as this being the case to do so, please provide more details if
> > you think your case cannot be solved taking in account new operations
> > are made from start each time.
> 
> I feel very strongly that a module should not have more than one 'view'
> of the database.  Each database should see the world as presented by
> those modules below it.  Anything more is going to simply cause
> confusion. 

Yes, and I think the view should be the view seen at the API level, not
some partial view seen down the path. This way all modules assumes the
same view.

You have some points in the discussion Andrew, I'm not deaf to your
claims, it's just that I think you underestimate the importance of some
modules and the fact that some modules may depend on each other so that
calling the stack from start does the right job and calling it from
inside not.

It may be more difficult to program a module keeping in mind all calls
start from top, but makes it easier to keep a consistent view at the API
level which we must assure.

I ma be wrong of course, but I do not see a definitive convincing
argument on your said, and my design still stand imho.

Simo.

-- 
Simo Sorce    -  idra at samba.org
Samba Team    -  http://www.samba.org
Italian Site  -  http://samba.xsec.it