[PATCH] s4-drs: replmd_delete implementation

[Eduardo Lima]

Hi Tridge,

My GIT tree was not updated.. I just did a commit with a newer (but not the
latest) version..

The version that is now on the git tree is the one that I was doing tests to
remove some fields from a deleted object.

Tomorrow I will ready more carefully your email.

Thanks!

--
Eduardo Lima
Sent from Campinas, SP, Brazil

On Wed, Nov 18, 2009 at 23:16, <tridge at samba.org> wrote:

> Hi Eduardo,
>
>  > I created this utility function that gets the defaultNamingContext and
> put
>  > "CN=Deleted Objects" in front of the DN.
>
> I found your git tree at git://repo.or.cz/Samba/eduardoll.git and had
> a look at the function. It has a couple of problems:
>
>  1) it suffers from the very common cut&paste problem with error
>  messages - I think 'dsServiceName' in the error message should be
>  'defaultNamingContext'
>
>  2) it doesn't add the "Deleted Objects" rDN to the returned
>  result. You could use ldb_dn_add_child_fmt() to add it like this:
>
>   deleted_dn = ldb_dn_add_child_fmt(base_dn, "Deleted Objects");
>
>  3) I don't think you actually need to do the search for
>  defaultNamingContext. Take a look at the function
>  samdb_partitions_dn() for what would be a closer example to what you
>  need. That one adds "CN=Partitions" to samdb_config_dn(). If you
>  instead add "CN=Deleted Objects" to samdb_base_dn() then I think
>  you'll have what you want.
>
>  > Will this work for every type of object? (every object in the
>  > domain - even if it's in the deepest depth - should be moved
>  > directly to CN=Deleted Objects,DC=w2k8... when deleted?)
>
> hmm, looking at a w2k8 server, it looks like there is a separate
> "Deleted Objects" container in the configuration partition. That means
> my suggestion above is wrong, and you'll instead need to work out the
> correct "Deleted Objects" location based upon the DN of the object
> being deleted.
>
> One curious thing is that there is no "Deleted Objects" container in
> the schema partition on w2k8. I think you should investigate what
> happens to deleted objects in the schema partition. Perhaps if a
> partition does not have a Deleted Objects container then
> the repl_meta_data module should not do the rename/modify change and
> instead should just delete it? Or maybe it get moved into the Deleted
> Objects container of the next partition up?
>
> You should investigate this by creating and then deleting an object in
> a schema partitino of a w2k8 server and see what happens to the
> object. If you run something like this:
>
>  bin/ldbsearch -H ldap://w2k8 -Uadministrator%password --controls
> show_deleted:1,search_options:1:2 'isDeleted=TRUE' dn objectclass
> lastKnownParent
>
> then it will give you an idea of where the deleted objects are going.
>
> Once you've done that, you'll need to add a function in
> repl_meta_data.c that works out the DN of the Deleted Objects
> container given the DN of an object being deleted. There are several
> ways to do this. One reasonable approach would be to add a function in
> dsdb/common/util.c like this:
>
>  struct ldb_dn *dsdb_find_nc_root(struct ldb_context *samdb, struct ldb_dn
> *dn);
>
> it would work by chopping off components from the DN (using
> ldb_dn_get_parent()) one at a time until it found a DN that has an
> instanceType with INSTANCE_TYPE_IS_NC_HEAD set. You'll need to do a
> base search for instanceType for each DN as you loop.
>
> Then you'd add the "CN=Deleted Objects" to that, and see if it
> exists. If it does then you can do the rename/modify. If not then just
> pass the original delete request down to the next module.
>
> You will also need to change our provision to add a "Deleted Objects"
> container for the configuration partition. If you look at
> setup/provision.ldif you'll see that it has adds "Deleted Objects"
> container for the DOMAINDN, but not for the configuration
> partition. You'll need to add it to
> setup/provision_configuration_basedn.ldif.
>
>  > This is the part that I'm having more difficulty. I still can't
> understand
>  > well what the modules are and the way that they work in the Samba
> source.  I
>  > saw that ldb_module structure has *prev and *next fields and modules
> might
>  > be linked together but why would all the modules be renamed if I use the
>  > ldb_rename with the top level ldb context? When I do a bin/ldbdel
> operation,
>  > what is the execution's flow that the modules follow?
>
> This is one of the more complex aspects of Samba. I'll try and explain
> it, and if you have any more questions then please ask.
>
> Plain ldb is very simple. It knows nothing about the complexities of
> active directory. It knows nothing about schemas, or objectclasses or
> any of the things that go to make up a AD compatible system. It just
> knows how to store, retrieve and modify objects in a dumb fashion.
>
> The way it becomes more like AD is to have a series of modules. If you
> do this search:
>
>  bin/ldbsearch -H $PREFIX/private/sam.ldb -b @MODULES -s base
>
> then you'll see the list of modules that are in your database. That
> list currently looks like this:
>
> @LIST:
> resolve_oids,rootdse,lazy_commit,paged_results,ranged_results,anr,server_sort,asq,extended_dn_store,extended_dn_in,rdn_name,objectclass,descriptor,acl,samldb,password_hash,operational,kludge_acl,instancetype,repl_meta_data,subtree_rename,subtree_delete,linked_attributes,extended_dn_out_ldb,show_deleted,schema_load,new_partition,partition
>
> as you can see, there are a lot of modules!
>
> When you call an ldb operation, for example a ldb_delete() call, then
> the ldb code will check with each of these modules in the order they
> are listed, and if the module has a registered 'delete' function then
> that function is called.
>
> If you look at the first module in the list (the resolve_oids module)
> and look near the bottom of resolve_oids.c you'll see that it doesn't
> have a function for delete. That means that the resolve_oids module
> will not affect delete operations.
>
> It does however have a method for 'add'. So let's look at what that
> does.
>
> Imagine we called ldb_add() to add an object to the database. The ldb
> code will see that the resolve_oids module has an 'add' method, so it
> will call resolve_oids_add(). If you look inside that function you'll
> see that it can complete in a number of ways:
>
>  1) if it decides that it doesn't need to do anything, it can call
>  ldb_next_request() passing the original request (the 'req'
>  argument). That asks ldb to call the next module in the list that has
>  an 'add' method.
>
>  2) it can return an error. You can see for examples places where it
>  returns LDB_ERR_OPERATIONS_ERROR when it fails to allocate some
>  memory.
>
>  3) it can construct a new request which does something different, and
>  then call ldb_next_request() with that new request. This is what it
>  does in this line:
>
>        return ldb_next_request(module, down_req);
>
>  That asks ldb to pass a newly constructed request down to the next
>  module. Notice that it has specified a callback
>  (resolve_oids_callback) that should be called when this new request
>  is finished.
>
> In this way, each ldb request passes through each of the modules that
> wants to be involved with that type of request. Eventually the request
> gets down to the last module in the chain, which is the ldb_tdb module
> that actually implements the backend (that is not shown in the @LIST
> above, it is implied). The ldb_tdb module, which just implements dumb
> message storage, does the actual change to the database.
>
> So, back to my original comment on using ldb_rename() with a ldb
> context. The way you had done it will work, but it probably isn't
> quite the right thing to do. You are calling it from within the
> repl_meta_data module. That means you are half way through the list of
> modules above. All of the modules above have either done what they
> want to do to this request, or don't want to do anything.
>
> By calling ldb_rename() you are starting at the top of the module list
> again. This means every module above you will see the rename. Do you
> want that? I think in this case you probably don't, and what you
> really want to do is pass the rename down to the modules below you,
> skipping the ones above.
>
> One of the reasons we try to do this is to avoid looping forever. If
> we go to the top of the stack for this rename, then that means the
> rename will come back down to the repl_meta_data module again. So
> we'll be calling our own module. I think in this case it will work,
> but it can be a bit hard to follow sometimes!
>
> If you look at dsdb/samdb/ldb_modules/util.c then you can see some
> helper functions have been added to make this sort of "do an operation
> on the rest of the modules" call a bit easier. The helper functions
> are currently only for search, but you could add a new one for doing a
> rename starting at the current module. Following the pattern in that
> file, you'd add a new function:
>
>  int dsdb_module_rename(struct ldb_module *module,
>                        struct ldb_dn *olddn, struct ldb_dn *newdn);
>
> it would call ldb_build_rename_req() to build a new rename request
> structure, then it would call ldb_next_request() and then call
> ldb_wait(), in much the same way that dsdb_module_search() works.
>
> If you then use dsdb_module_rename() in your new code, then you will
> be doing the rename only on the modules below you in the stack. You
> should similarly add a dsdb_module_modify() call that follows the same
> pattern.
>
> The only caveat to this is if there is a module above you in the stack
> that you actually want to see the rename. For example, the rdn_name
> module (in lib/ldb/modules/rdn_name.c) has special handling for rename
> operations to fix the 'name' field. If you follow my suggestion above
> then this special handling won't happen. What you need to do is check
> on a w2k8 server and see if the 'name' attribute is changed when you
> delete an object.
>
> If I do this:
>
>  bin/ldbsearch -H ldap://w2k8 -Uadministrator%password --controls
> show_deleted:1,search_options:1:2 'isDeleted=TRUE' dn objectclass
> lastKnownParent name --show-binary
>
> then I can see that, yes, the 'name' attribute does look like it is
> being modified when you do a delete. That means we have two choices:
>
>  1) also fix the 'name' attribute in the modify operation that you
>  will do as part of the delete handling in repl_meta_data.c. Add it
>  to the same modify that adds the isDeleted=TRUE attribute.
>
>  2) forget what I said above and just use ldb_rename(). That means
>  that the rdn_name modules will do the fixup of 'name' for you.
>
> I think that (1) is a better choice, as I think it makes it clearer
> exactly what a delete does in the repl_meta_data module.
>
> I hope this helps!
>
> Cheers, Tridge
>