[SCM] Samba Shared Repository - branch master updated

Matthieu Patou mat at samba.org
Sat May 21 07:41:01 MDT 2011


The branch, master has been updated
       via  850cca3 add a demo script for dirsync
       via  c2fa348 s4-dsdb: add unit tests for dirsync control
       via  fa400af s4-dsdb: implementation of the dirsync control
       via  7b4e1e7 s4-dsdb: introduce dsdb_module_search_tree
       via  37b1662 s4-dsdb: relax a bit the checks on read acl when dirsync control is specified
       via  1d0fc44 s4-dsdb: create flag for requesting ACL relax in case of DIRSYNC request
       via  df83e9c s4: do not change the critical flag when it's on a dirsync control
       via  fae229a selftest: Allow to test samba4 with ACL on read set
      from  49c99d0 s4: add blackbox test for rename

http://gitweb.samba.org/?p=samba.git;a=shortlog;h=master


- Log -----------------------------------------------------------------
commit 850cca3996605a45d3f203318a90d3941934a792
Author: Matthieu Patou <mat at matws.net>
Date:   Mon Feb 7 09:55:26 2011 +0300

    add a demo script for dirsync
    
    Signed-off-by: Andrew Tridgell <tridge at samba.org>
    
    Autobuild-User: Matthieu Patou <mat at samba.org>
    Autobuild-Date: Sat May 21 15:40:26 CEST 2011 on sn-devel-104

commit c2fa3488317fa6b5eac0c7acc1fbed1e8cebcb7d
Author: Matthieu Patou <mat at matws.net>
Date:   Sun Feb 27 12:24:45 2011 +0300

    s4-dsdb: add unit tests for dirsync control
    
    Signed-off-by: Andrew Tridgell <tridge at samba.org>

commit fa400af18b7fdba2edd6c3b4c335dab64481545a
Author: Matthieu Patou <mat at matws.net>
Date:   Thu Jan 13 21:55:11 2011 +0300

    s4-dsdb: implementation of the dirsync control
    
    Signed-off-by: Andrew Tridgell <tridge at samba.org>

commit 7b4e1e78bef7ece31ee01ef3a1d7cfc3810ab27d
Author: Matthieu Patou <mat at matws.net>
Date:   Sun Mar 20 01:20:22 2011 +0300

    s4-dsdb: introduce dsdb_module_search_tree
    
    With this function your own search tree can be specified
    
    This function is similar to ldb_build_search_req_ex as it allows to
    pass a parse tree structure.
    
    Signed-off-by: Andrew Tridgell <tridge at samba.org>

commit 37b1662a38259d59508faa1b6226406b02504a5b
Author: Matthieu Patou <mat at matws.net>
Date:   Tue Mar 8 01:02:32 2011 +0300

    s4-dsdb: relax a bit the checks on read acl when dirsync control is specified
    
    Signed-off-by: Andrew Tridgell <tridge at samba.org>

commit 1d0fc445fae4b908ac475d0beb5e1d8d14a3efcb
Author: Matthieu Patou <mat at matws.net>
Date:   Sat Apr 16 11:46:40 2011 +0400

    s4-dsdb: create flag for requesting ACL relax in case of DIRSYNC request
    
    Signed-off-by: Andrew Tridgell <tridge at samba.org>

commit df83e9c15e93502e77cfa877d6bdf1beb9b4048f
Author: Matthieu Patou <mat at matws.net>
Date:   Mon Feb 7 09:58:17 2011 +0300

    s4: do not change the critical flag when it's on a dirsync control
    
    Signed-off-by: Andrew Tridgell <tridge at samba.org>

commit fae229aa3df470cf9cd87b42d60aa6be4211d1ae
Author: Matthieu Patou <mat at matws.net>
Date:   Thu Apr 14 09:48:14 2011 +0400

    selftest: Allow to test samba4 with ACL on read set
    
    Signed-off-by: Andrew Tridgell <tridge at samba.org>

-----------------------------------------------------------------------

Summary of changes:
 selftest/target/Samba4.pm                    |    3 +
 source4/dsdb/samdb/ldb_modules/acl_read.c    |   54 +-
 source4/dsdb/samdb/ldb_modules/dirsync.c     | 1359 ++++++++++++++++++++++++++
 source4/dsdb/samdb/ldb_modules/rootdse.c     |    6 +-
 source4/dsdb/samdb/ldb_modules/samba_dsdb.c  |    1 +
 source4/dsdb/samdb/ldb_modules/util.c        |   87 ++-
 source4/dsdb/samdb/ldb_modules/wscript_build |    9 +
 source4/dsdb/samdb/samdb.h                   |    1 +
 source4/dsdb/tests/python/dirsync.py         |  713 ++++++++++++++
 source4/scripting/devel/demodirsync.py       |  156 +++
 source4/selftest/knownfail                   |    2 +
 source4/selftest/tests.py                    |    1 +
 12 files changed, 2355 insertions(+), 37 deletions(-)
 create mode 100644 source4/dsdb/samdb/ldb_modules/dirsync.c
 create mode 100755 source4/dsdb/tests/python/dirsync.py
 create mode 100755 source4/scripting/devel/demodirsync.py


Changeset truncated at 500 lines:

diff --git a/selftest/target/Samba4.pm b/selftest/target/Samba4.pm
index bbe64a9..d27ee41 100644
--- a/selftest/target/Samba4.pm
+++ b/selftest/target/Samba4.pm
@@ -578,8 +578,11 @@ sub provision_raw_step1($$)
 		warn("can't open $ctx->{smb_conf}$?");
 		return undef;
 	}
+	my $acl = "false";
+	$acl = "true" if (defined $ENV{WITH_ACL});
 	print CONFFILE "
 [global]
+	acl:search = $acl
 	netbios name = $ctx->{netbiosname}
 	posix:eadb = $ctx->{lockdir}/eadb.tdb
 	workgroup = $ctx->{domain}
diff --git a/source4/dsdb/samdb/ldb_modules/acl_read.c b/source4/dsdb/samdb/ldb_modules/acl_read.c
index 181619a..35a840e 100644
--- a/source4/dsdb/samdb/ldb_modules/acl_read.c
+++ b/source4/dsdb/samdb/ldb_modules/acl_read.c
@@ -47,6 +47,7 @@ struct aclread_context {
 	bool sd;
 	bool instance_type;
 	bool object_sid;
+	bool indirsync;
 };
 
 struct aclread_private {
@@ -158,18 +159,41 @@ static int aclread_callback(struct ldb_request *req, struct ldb_reply *ares)
 							     access_mask,
 							     attr);
 
-			 if (ret == LDB_ERR_INSUFFICIENT_ACCESS_RIGHTS) {
-				 /* do not return this entry if attribute is
-				    part of the search filter */
-				 if (dsdb_attr_in_parse_tree(ac->req->op.search.tree,
-							     msg->elements[i].name)) {
-					 talloc_free(tmp_ctx);
-					 return LDB_SUCCESS;
-				 }
-				 aclread_mark_inaccesslible(&msg->elements[i]);
-			 } else if (ret != LDB_SUCCESS) {
-				 goto fail;
-			 }
+			/*
+			 * Dirsync control needs the replpropertymetadata attribute
+			 * so return it as it will be removed by the control
+			 * in anycase.
+			 */
+			if (ret == LDB_ERR_INSUFFICIENT_ACCESS_RIGHTS) {
+				if (!ac->indirsync) {
+					/* do not return this entry if attribute is
+					part of the search filter */
+					if (dsdb_attr_in_parse_tree(ac->req->op.search.tree,
+								msg->elements[i].name)) {
+						talloc_free(tmp_ctx);
+						return LDB_SUCCESS;
+					}
+					aclread_mark_inaccesslible(&msg->elements[i]);
+				} else {
+					/*
+					 * We are doing dirysnc answers
+					 * and the object shouldn't be returned (normally)
+					 * but we will return it without replPropertyMetaData
+					 * so that the dirysync module will do what is needed
+					 * (remove the object if it is not deleted, or return
+					 * just the objectGUID if it's deleted).
+					 */
+					if (dsdb_attr_in_parse_tree(ac->req->op.search.tree,
+								msg->elements[i].name)) {
+						ldb_msg_remove_attr(msg, "replPropertyMetaData");
+						break;
+					} else {
+						aclread_mark_inaccesslible(&msg->elements[i]);
+					}
+				}
+			} else if (ret != LDB_SUCCESS) {
+				goto fail;
+			}
 		 }
 		 for (i=0; i < msg->num_elements; i++) {
 			 if (!aclread_is_inaccessible(&msg->elements[i])) {
@@ -224,6 +248,7 @@ static int aclread_search(struct ldb_module *module, struct ldb_request *req)
 	struct aclread_context *ac;
 	struct ldb_request *down_req;
 	struct ldb_control *as_system = ldb_request_get_control(req, LDB_CONTROL_AS_SYSTEM_OID);
+	uint32_t flags = ldb_req_get_custom_flags(req);
 	struct ldb_result *res;
 	struct aclread_private *p;
 	bool is_untrusted = ldb_req_is_untrusted(req);
@@ -284,6 +309,11 @@ static int aclread_search(struct ldb_module *module, struct ldb_request *req)
 	ac->module = module;
 	ac->req = req;
 	ac->schema = dsdb_get_schema(ldb, req);
+	if (flags & DSDB_ACL_CHECKS_DIRSYNC_FLAG) {
+		ac->indirsync = true;
+	} else {
+		ac->indirsync = false;
+	}
 	if (!ac->schema) {
 		return ldb_operr(ldb);
 	}
diff --git a/source4/dsdb/samdb/ldb_modules/dirsync.c b/source4/dsdb/samdb/ldb_modules/dirsync.c
new file mode 100644
index 0000000..64c5047
--- /dev/null
+++ b/source4/dsdb/samdb/ldb_modules/dirsync.c
@@ -0,0 +1,1359 @@
+/*
+   SAMDB control module
+
+   Copyright (C) Matthieu Patou <mat at matws.net> 2011
+
+   This program is free software; you can redistribute it and/or modify
+   it under the terms of the GNU General Public License as published by
+   the Free Software Foundation; either version 3 of the License, or
+   (at your option) any later version.
+
+   This program is distributed in the hope that it will be useful,
+   but WITHOUT ANY WARRANTY; without even the implied warranty of
+   MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+   GNU General Public License for more details.
+
+   You should have received a copy of the GNU General Public License
+   along with this program.  If not, see <http://www.gnu.org/licenses/>.
+*/
+
+
+#include "includes.h"
+#include "ldb/include/ldb.h"
+#include "ldb/include/ldb_errors.h"
+#include "ldb/include/ldb_module.h"
+#include "libcli/security/security.h"
+#include "librpc/gen_ndr/drsblobs.h"
+#include "librpc/gen_ndr/ndr_drsblobs.h"
+#include "librpc/ndr/libndr.h"
+#include "dsdb/samdb/samdb.h"
+#include "util.h"
+
+#define LDAP_DIRSYNC_OBJECT_SECURITY		0x01
+#define LDAP_DIRSYNC_ANCESTORS_FIRST_ORDER	0x800
+#define LDAP_DIRSYNC_PUBLIC_DATA_ONLY		0x2000
+#define LDAP_DIRSYNC_INCREMENTAL_VALUES		0x80000000
+
+
+struct dirsync_context {
+	struct ldb_module *module;
+	struct ldb_request *req;
+
+	/*
+	 * We keep a track of the number of attributes that we
+	 * add just for the need of the implementation
+	 * it will be usefull to track then entries that needs not to
+	 * be returned because there is no real change
+	 */
+
+	unsigned int nbDefaultAttrs;
+	uint64_t highestUSN;
+	uint64_t fromreqUSN;
+	uint32_t cursor_size;
+	bool noextended;
+	bool linkIncrVal;
+	bool localonly;
+	bool partial;
+	bool assystem;
+	int functional_level;
+	const struct GUID *our_invocation_id;
+	const struct dsdb_schema *schema;
+	struct ldb_dn *nc_root;
+	struct drsuapi_DsReplicaCursor *cursors;
+};
+
+
+static int dirsync_filter_entry(struct ldb_request *req,
+					struct ldb_message *msg,
+					struct ldb_control **controls,
+					struct dirsync_context *dsc,
+					bool referral)
+{
+	struct ldb_context *ldb;
+	uint64_t val;
+	enum ndr_err_code ndr_err;
+	uint32_t n;
+	int i;
+	unsigned int size, j;
+	uint32_t deletedattr;
+	struct ldb_val *replMetaData = NULL;
+	struct replPropertyMetaDataBlob rmd;
+	const struct dsdb_attribute *attr;
+	const char **listAttr = NULL;
+	bool namereturned = false;
+	bool nameasked = false;
+	NTSTATUS status;
+	/* Ajustment for the added attributes, it will reduce the number of
+	 * expected to be here attributes*/
+	unsigned int delta = 0;
+	const char **myaccept = NULL;
+	const char *emptyaccept[] = { NULL };
+	const char *extendedaccept[] = { "GUID", "SID", "WKGUID", NULL };
+	const char *rdn = NULL;
+	struct ldb_message_element *el;
+	struct ldb_message *newmsg;
+	bool keep = false;
+	/*
+	 * Where we asked to do extended dn ?
+	 * if so filter out everything bug GUID, SID, WKGUID,
+	 * if not filter out everything (just keep the dn).
+	 */
+	if ( dsc->noextended == true ) {
+		myaccept = emptyaccept;
+	} else {
+		myaccept = extendedaccept;
+	}
+	ldb = ldb_module_get_ctx(dsc->module);
+
+	if (msg->num_elements == 0) {
+		/*
+			* Entry that we don't really have access to
+			*/
+		return LDB_SUCCESS;
+	}
+	ldb_dn_extended_filter(msg->dn, myaccept);
+
+	/*
+	* If the RDN starts with CN then the CN attribute is never returned
+	*/
+	rdn = ldb_dn_get_rdn_name(msg->dn);
+
+	deletedattr = 0;
+	/*
+	 * if objectGUID is asked and we are dealing for the referrals entries and
+	 * the usn searched is 0 then we didn't count the objectGUID as an automatically
+	 * returned attribute, do to so we increament delta.
+	 */
+	if (referral == true &&
+			ldb_attr_in_list(req->op.search.attrs, "objectGUID") &&
+			dsc->fromreqUSN == 0) {
+		delta++;
+	}
+
+
+	/*
+	 * In terms of big O notation this is not the best algorithm,
+	 * but we try our best not to make the worse one.
+	 * We are obliged to run through the n message's elements
+	 * and through the p elements of the replPropertyMetaData.
+	 *
+	 * It turns out that we are crawling twice the message's elements
+	 * the first crawl is to remove the non replicated and generated
+	 * attributes. The second one is to remove attributes that haven't
+	 * a USN > as the requested one.
+	 *
+	 * In the second crawl we are reading the list of elements in the
+	 * replPropertyMetaData for each remaining replicated attribute.
+	 * In order to keep the list small
+	 *
+	 * We have a O(n'*p') complexity, in worse case n' = n and p' = p
+	 * but in most case n' = n/2 (at least half of returned attributes
+	 * are not replicated or generated) and p' is small as we
+	 * list only the attribute that have been modified since last interogation
+	 *
+	 */
+	newmsg = talloc_zero(dsc->req, struct ldb_message);
+	if (newmsg == NULL) {
+		return ldb_oom(ldb);
+	}
+	for (i = msg->num_elements - 1; i >= 0; i--) {
+		attr = dsdb_attribute_by_lDAPDisplayName(dsc->schema, msg->elements[i].name);
+		if (ldb_attr_cmp(msg->elements[i].name, "uSNChanged") == 0) {
+			/* Read the USN it will used at the end of the filtering
+			 * to update the max USN in the cookie if we
+			 * decide to keep this entry
+			 */
+			val = strtoull((const char*)msg->elements[i].values[0].data, NULL, 0);
+			continue;
+		}
+
+		if (ldb_attr_cmp(msg->elements[i].name,
+						"replPropertyMetaData") == 0) {
+			replMetaData = (talloc_steal(dsc, &msg->elements[i].values[0]));
+			continue;
+		}
+	}
+
+	if (replMetaData == NULL) {
+		bool guidfound = false;
+
+		/*
+		 * We are in the case of deleted object where we don't have the
+		 * right to read it.
+		 */
+		if (!ldb_msg_find_attr_as_uint(msg, "isDeleted", 0)) {
+			/*
+			 * This is not a deleted item and we don't
+			 * have the replPropertyMetaData.
+			 * Do not return it
+			 */
+			return LDB_SUCCESS;
+		}
+		newmsg->dn = ldb_dn_new(newmsg, ldb, "");
+		if (newmsg->dn == NULL) {
+			return ldb_oom(ldb);
+		}
+
+		el = ldb_msg_find_element(msg, "objectGUID");
+		if ( el != NULL) {
+			guidfound = true;
+		}
+		/*
+		 * We expect to find the GUID in the object,
+		 * if it turns out not to be the case sometime
+		 * well will uncomment the code bellow
+		 */
+		SMB_ASSERT(guidfound == true);
+		/*
+		if (guidfound == false) {
+			struct GUID guid;
+			struct ldb_val *new_val;
+			DATA_BLOB guid_blob;
+
+			tmp[0] = '\0';
+			txt = strrchr(txt, ':');
+			if (txt == NULL) {
+				return ldb_module_done(dsc->req, NULL, NULL, LDB_ERR_OPERATIONS_ERROR);
+			}
+			txt++;
+
+			status = GUID_from_string(txt, &guid);
+			if (!NT_STATUS_IS_OK(status)) {
+				return ldb_module_done(dsc->req, NULL, NULL, LDB_ERR_OPERATIONS_ERROR);
+			}
+
+			status = GUID_to_ndr_blob(&guid, msg, &guid_blob);
+			if (!NT_STATUS_IS_OK(status)) {
+				return ldb_module_done(dsc->req, NULL, NULL, LDB_ERR_OPERATIONS_ERROR);
+			}
+
+			new_val = talloc(msg, struct ldb_val);
+			if (new_val == NULL) {
+				return ldb_oom(ldb);
+			}
+			new_val->data = talloc_steal(new_val, guid_blob.data);
+			new_val->length = guid_blob.length;
+			if (ldb_msg_add_value(msg, "objectGUID", new_val, NULL) != 0) {
+				return ldb_module_done(dsc->req, NULL, NULL, LDB_ERR_OPERATIONS_ERROR);
+			}
+		}
+		*/
+		ldb_msg_add(newmsg, el, LDB_FLAG_MOD_ADD);
+		talloc_steal(newmsg->elements, el->name);
+		talloc_steal(newmsg->elements, el->values);
+
+		talloc_free(msg);
+		return ldb_module_send_entry(dsc->req, msg, controls);
+	}
+
+	ndr_err = ndr_pull_struct_blob(replMetaData, dsc, &rmd,
+		(ndr_pull_flags_fn_t)ndr_pull_replPropertyMetaDataBlob);
+	if (!NDR_ERR_CODE_IS_SUCCESS(ndr_err)) {
+		ldb_set_errstring(ldb, "Unable to unmarshall replPropertyMetaData");
+		return ldb_module_done(dsc->req, NULL, NULL, LDB_ERR_OPERATIONS_ERROR);
+	}
+	if (ldb_attr_in_list(req->op.search.attrs, "name") ||
+			ldb_attr_in_list(req->op.search.attrs, "*")) {
+		nameasked = true;
+	}
+
+	/*
+		* If we don't have an USN and no updateness array then we skip the
+		* test phase this is an optimisation for the case when you
+		* first query the DC without a cookie.
+		* As this query is most probably the one
+		* that will return the biggest answer, skipping this part
+		* will really save time.
+		*/
+	if (ldb_dn_compare(dsc->nc_root, msg->dn) == 0) {
+		/* If we have name then we expect to have parentGUID,
+		 * it will not be the case for the root of the NC
+		 */
+		delta++;
+	}
+
+	if (dsc->fromreqUSN > 0 || dsc->cursors != NULL) {
+		j = 0;
+		/*
+		* Allocate an array of size(replMetaData) of char*
+		* we know that it will be oversized but it's a short lived element
+		*/
+		listAttr = talloc_array(msg, const char*, rmd.ctr.ctr1.count + 1);
+		if (listAttr == NULL) {
+			return ldb_oom(ldb);
+		}
+		for (n=0; n < rmd.ctr.ctr1.count; n++) {
+			struct replPropertyMetaData1 *omd = &rmd.ctr.ctr1.array[n];
+			if (omd->local_usn > dsc->fromreqUSN) {
+				const struct dsdb_attribute *a = dsdb_attribute_by_attributeID_id(dsc->schema,
+										omd->attid);
+				if (!dsc->localonly) {
+					struct drsuapi_DsReplicaCursor *tab = dsc->cursors;
+					uint32_t l;
+					for (l=0; l < dsc->cursor_size; l++) {
+						if (GUID_equal(&tab[l].source_dsa_invocation_id, &omd->originating_invocation_id) &&
+								tab[l].highest_usn >= omd->originating_usn) {
+							/*
+							 * If we have in the uptodateness vector an entry
+							 * with the same invocation id as the originating invocation
+							 * and if the usn in the vector is greater or equal to
+							 * the one in originating_usn, then it means that this entry
+							 * has already been sent (from another DC) to the client
+							 * no need to resend it one more time.
+							 */
+							goto skip;
+						}
+					}
+					/* If we are here it's because we have a usn > (max(usn of vectors))*/
+				}
+				if (namereturned == false &&
+						nameasked == true &&
+						ldb_attr_cmp(a->lDAPDisplayName, "name") == 0) {
+					namereturned = true;
+					if (ldb_dn_compare(dsc->nc_root, msg->dn) == 0) {
+						delta++;
+					}
+				}
+				listAttr[j] = a->lDAPDisplayName;
+				j++;
+skip:
+				continue;
+			}
+		}
+		size = j;
+	} else {
+		size = 0;
+		if (ldb_attr_in_list(req->op.search.attrs, "*") ||
+				ldb_attr_in_list(req->op.search.attrs, "name")) {
+			namereturned = true;
+		}
+	}
+
+
+	/*
+	 * Let's loop around the remaining elements
+	 * to see which one are in the listAttr.
+	 * If they are in this array it means that
+	 * their localusn > usn from the request (in the cookie)
+	 * if not we remove the attribute.
+	 */
+	for (i = msg->num_elements - 1; i >= 0; i--) {
+		el = &(msg->elements[i]);
+		attr = dsdb_attribute_by_lDAPDisplayName(dsc->schema,
+				el->name);
+		const char *ldapattrname = el->name;
+		keep = false;
+
+		if (attr->linkID & 1) {
+			/*
+			 * Attribute is a backlink so let's remove it
+			 */
+			continue;
+		}
+
+		if (ldb_attr_cmp(msg->elements[i].name,
+						"replPropertyMetaData") == 0) {
+			continue;
+		}
+
+		if ((attr->systemFlags & (DS_FLAG_ATTR_NOT_REPLICATED | DS_FLAG_ATTR_IS_CONSTRUCTED))) {
+			if (ldb_attr_cmp(attr->lDAPDisplayName, "objectGUID") != 0 &&
+					ldb_attr_cmp(attr->lDAPDisplayName, "parentGUID") != 0) {
+				/*
+				 * Attribute is constructed or not replicated, let's get rid of it
+				 */
+				continue;
+			} else {
+				/* Let's keep the attribute that we forced to be added
+				 * even if they are not in the replicationMetaData
+				 * or are just generated
+				 */
+				if (namereturned == false &&
+					(ldb_attr_cmp(attr->lDAPDisplayName, "parentGUID") == 0)) {
+					delta++;
+					continue;
+				}
+				if (ldb_msg_add(newmsg, el, LDB_FLAG_MOD_ADD) != LDB_SUCCESS) {
+					return ldb_error(ldb,
+						LDB_ERR_OPERATIONS_ERROR,
+						"Unable to add attribute");
+				}
+				talloc_steal(newmsg->elements, el->name);
+				talloc_steal(newmsg->elements, el->values);
+				continue;
+			}
+		}
+
+		if (ldb_attr_cmp(msg->elements[i].name, rdn) == 0) {
+			/*
+			 * We have an attribute that is the same as the start of the RDN
+			 * (ie. attribute CN with rdn CN=).
+			 */
+			continue;


-- 
Samba Shared Repository


More information about the samba-cvs mailing list