[SCM] Samba Shared Repository - branch master updated
Douglas Bagnall
dbagnall at samba.org
Thu Feb 9 06:08:03 UTC 2017
The branch, master has been updated
via 45df61e Switch on the sortedLinks Flag on new databases
via 84d0862 replmd: check for the sortedLinks feature flag
via fea92e5 dsdb: Honour @SAMBA_FEATURES_SUPPORTED flag in @IDXATTR
via 48da42c schema: Set flag into @INDEXLIST to indicate we support feature flags
via 8368e06 samba_dsdb: Use and maintain compatibleFeatures and requiredFeatures in @SAMBA_DSDB
via 538bbd5 repl_md: get links in sorted order in replmd_add_fix_la
via 33a5b6a replmd: treat a zero GUID as not present in get_parsed_dns
via 5fa1b5c replmd: keep links sorted in replmd_process_linked_attribute
via 762bde2 replmd linked_attributes: maintain sorted links in replace
via edcfede replmd linked attributes: use really_parse_trusted_dn everywhere
via ecf6e13 replmd: simplify and optimise replmd_modify_la_delete
via 95bc1e0 replmd: rearrange nothing-to-delete logic
via 976a213 repl_meta_data: linked attributes use DRS sort order
via bd6b417 replmd: rework replmd_modify_la_add to merge efficiently
via 06cca52 replmd linked attrs: fully parse dn for upgrade check
via 47efee0 replmd linked attributes: lazy parsing for trusted DNs
via ffff8c8 replmd: Add placeholder sorted_links to struct replmd_private
via c1bceb2 replmd: replmd_check_upgrade_links() needs to first parses DNs
via 33b2758 replmd: parsed_dn_find() finds insertion point as well as exact hit
via 83c4ad7 binsearch: make BINARY_ARRAY_SEARCH_GTE compare against a pointer
via 8bdec70 binsearch: clarify variable name in greater-than-or-equal search
via 3c9483f replmd: fix variable names in replmd_check_upgrade_links
via 5f52a8f replmd: replmd_check_upgrade_links() only checks the first DN
via 225fa43 replmd: pass replmd_private down to replmd_add_backlink()
via 10e8bc1 replmd: Fix some whitespace in repl_meta_data.c
via 996aafd replmd: check whether list is already sorted in get_parsed_dns()
via a9e0e7a selftest: Do not test for link ordering in tombstones_expunge test
via 2fab758 s4/linked_attribute tests: remove helper function unused parameter
via d3dec7a s4/linked_attribute tests: try adding linked attributes directly
via b69f48e s4/linked_attribute tests: test with the relax control
via 6bee3fa s4/linked_attribute tests: compare link lists in sorted order
via 069c4ad s4/linked_attribute tests: remove unused code
via 1647f0c s4/linked_attribute tests: add multiple links and replace tests
via 3502928 s4/linked_attributes test: pep8 tidy-up, remove unused imports
from 5c918e2 torture/drs: expand test for DRSUAPI_DRS_GET_ANC
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 45df61e943696d84466e6379da1b28c6a4d50e87
Author: Andrew Bartlett <abartlet at samba.org>
Date: Wed Jan 11 09:24:06 2017 +1300
Switch on the sortedLinks Flag on new databases
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Autobuild-User(master): Douglas Bagnall <dbagnall at samba.org>
Autobuild-Date(master): Thu Feb 9 07:07:43 CET 2017 on sn-devel-144
commit 84d08629af2e5de1da209023be484cdf19570579
Author: Andrew Bartlett <abartlet at samba.org>
Date: Fri Feb 3 11:25:37 2017 +1300
replmd: check for the sortedLinks feature flag
If it is there, we assume linked attributes are stored in a sorted
order.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
commit fea92e5f1d57909df392e237129dfd2091f63555
Author: Andrew Bartlett <abartlet at samba.org>
Date: Fri Feb 3 11:47:41 2017 +1300
dsdb: Honour @SAMBA_FEATURES_SUPPORTED flag in @IDXATTR
This allows us to detect modification by a Samba version prior to
the introduction of the compatibleFeatures logic as this flag will
be stripped by the schema load code of older Samba versions.
Therefore if it is not present, then remove all
compatibleFeatures.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Pair-programmed-with: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 48da42c85f2924f4526919825bb3dec0f2259d7e
Author: Andrew Bartlett <abartlet at samba.org>
Date: Fri Feb 3 16:13:43 2017 +1300
schema: Set flag into @INDEXLIST to indicate we support feature flags
Because @INDEXLIST is rewritten by all Samba versions, we can detect
that we have opened the database with an older version that does not
support the feature flags by the absense of this in @INDEXLIST
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
commit 8368e06ff65cc70e1cf13a0eb4349033e068fcc6
Author: Andrew Bartlett <abartlet at samba.org>
Date: Thu Jan 12 16:51:45 2017 +1300
samba_dsdb: Use and maintain compatibleFeatures and requiredFeatures in @SAMBA_DSDB
This will allow us to introduce new database features that are
backward compatible from the point of view of older versions of Samba,
but which will be damaged by modifying the database with such a
version.
For example, if linked attributes are stored in sorted order in 4.7,
and this change, without any values in current_supportedFeatures is
itself included in 4.6, then our sortedLinks are backward compatible
to that release.
That is with 4.6 (including this patch) which doesn't care about
ordering -- but a downgraded 4.7 database used by 4.6 will be broken
when later used with 4.7. If we add a 'sortedLinks' feature flag in
compatibleFeatures, we can detect that.
This will allow us to determine if the database still contains
unsorted links, as that information allows us to make the code
handling links much more efficient.
We won't add the actual flag until all the code is in place.
Andrew wrote the actual code and Douglas wrote the tests, and they
cross-reviewed.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Piar-programmed-with: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
selftest: check for database features flags
commit 538bbd5e9c492f606215ae55e44aa4bdc22f8a9f
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Wed Feb 1 17:34:51 2017 +1300
repl_md: get links in sorted order in replmd_add_fix_la
This is where forward links get added when they get added with an
object.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 33a5b6a934ea5098cd84d3caaddc7658738c9d5e
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Thu Feb 2 16:37:58 2017 +1300
replmd: treat a zero GUID as not present in get_parsed_dns
This roughly follows the pattern in the 2009 commit
0d5d7f58473c989bff4 by the Andrews Tridgell and Bartlett, which dealt
with zero GUIDs in replmd_add_fix_la(). That function is about to use
get_parsed_dns() [see next commit], and the other users of
get_parsed_dns don't really want to see zero guids, so it is simpler
to test here.
This makes hitting the GUID_all_zero branch of parsed_dn_find() even
more unlikely.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 5fa1b5cc79d813435eefe912bd2530de70fa1baf
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Fri Jan 27 17:46:22 2017 +1300
replmd: keep links sorted in replmd_process_linked_attribute
This is where linked attributes get added during a replication.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Pair-programmed-with: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 762bde22ed27866c2e1e7de5181645532814a170
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Fri Jan 6 16:38:03 2017 +1300
replmd linked_attributes: maintain sorted links in replace
We use a merge-like algorithm, which gives us a slight algorithmic
improvement (O(m + n) vs O(m log(n) + n log(m))) and keeps the results
sorted.
Here's an example. There are existing links to {A C D* F*} where D*
and F* represent deleted links, and we want to replace them with {B C
E F}.
existing: A C D* E F*
| | |
replacements: B C E F
result: A* B C D* E F
This is what happens to each link:
A gets deleted to A*.
B gets added.
C is retained, with possible extended DN changes.
D* stays in the list as a deleted link
E is retained like C
F is undeleted.
Backlinks are created in the case of B and F
The backlink for A is deleted
The backlinks are not changed for C and E or D* (D* has none)
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit edcfededd2f92e7d0196a077890c69a07b2d887b
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Wed Jan 11 17:49:24 2017 +1300
replmd linked attributes: use really_parse_trusted_dn everywhere
This function fills out the DN and GUID fields of an unparsed
parsed_dn struct, which was happening in a few other places already.
In some places the GUID was not being filled out, which would probably
cause problems if the sorted_links switch was turned on.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit ecf6e13749f779d6f56c13951e3d48fdcab18820
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Fri Jan 6 11:00:57 2017 +1300
replmd: simplify and optimise replmd_modify_la_delete
With the old binary search, we didn't get a pointer to the found
value, just a yes or no answer as to its existence. That meant we
ended up searching in both directions to find the links to be deleted.
As a consequence we needed to parse out the GUID of every existing
link, even if it wasn't being deleted.
Here we do it in one pass.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 95bc1e0640c6071e451b2a029c91fa18a3db0294
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Fri Jan 6 09:49:38 2017 +1300
replmd: rearrange nothing-to-delete logic
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 976a213b1821726842a3ca3d0b0db7d575c4c5bd
Author: Andrew Bartlett <abartlet at samba.org>
Date: Wed Jan 4 14:09:00 2017 +1300
repl_meta_data: linked attributes use DRS sort order
Links come over the wire as if sorted by memcmp() on the binary blobs,
not as sorted by GUID_compare(). Until a few patches ago, a newly
joined DC would have its linked attributes in the memcmp order. This
restores that behaviour.
This comparison could be made more efficient by storing the GUID in
the original state, but it does not seem to be a bottleneck.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Pair-programmed-with: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
commit bd6b417179938dc89e16672d2290901463355b16
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Thu Dec 29 13:20:41 2016 +1300
replmd: rework replmd_modify_la_add to merge efficiently
Because both the list of added links and the list of existing links
are sorted, it is possible to interlace the two and obtain a merged
sorted list.
We avoid a great amount of talloc_realloc()ing by observing that the
merged list can't be longer than the sum of the two lists.
In the (common) case where there are many existing links but few being
added, we avoid parsing most of the existing link DNs and GUIDs if the
sorted_links feature flag is set.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Pair-programmed-with: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 06cca52d7824f6615a67938c742236cf6482c811
Author: Andrew Bartlett <abartlet at samba.org>
Date: Fri Feb 3 15:35:02 2017 +1300
replmd linked attrs: fully parse dn for upgrade check
Elsewhere we use the dsdb_dn pointer as a flag indicating parsed-ness,
so we have to be consistent.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 47efee074a82a5f203652302f1fe7a66cdfa7b34
Author: Andrew Bartlett <abartlet at samba.org>
Date: Fri Feb 3 15:34:17 2017 +1300
replmd linked attributes: lazy parsing for trusted DNs
If we know that links from the database are in sorted order (via the
replmd_private->sorted_links flag), we can avoid actually parsing them
until it is absolutely necessary.
In many cases we are adding a single link to a long list. The location
of the single link is found via a binary search, so we end up parsing
log(N) DNs instead of N.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit ffff8c847e0beae7c1b29d3b8010eb300f80b9fa
Author: Andrew Bartlett <abartlet at samba.org>
Date: Fri Feb 3 15:31:55 2017 +1300
replmd: Add placeholder sorted_links to struct replmd_private
This will be initialised to false (zero) by default and will later come
from the compatibleFeatures in @SAMBA_DSDB
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
commit c1bceb2528a6c1bdaec39d9d87cc34c1ef8c28df
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Thu Dec 29 15:09:15 2016 +1300
replmd: replmd_check_upgrade_links() needs to first parses DNs
Because we now load the dns with get_parsed_dns_trusted we have
to manually explode them in the upgrade tests.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 33b2758c5533d178570a94e88815af0278031e30
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Thu Dec 29 12:12:23 2016 +1300
replmd: parsed_dn_find() finds insertion point as well as exact hit
This will allow us to maintain the list of links in sorted order.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 83c4ad778e6ffcd3e672a068a8e62f1611531298
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Thu Dec 22 16:09:22 2016 +1300
binsearch: make BINARY_ARRAY_SEARCH_GTE compare against a pointer
This is in preparation for improvements in our handling of linked
attributes where we make changes to the pointer in the process of
comparing it (for caching purposes).
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 8bdec7034e121592fa9faec70bd951319334e560
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Thu Dec 15 14:39:33 2016 +1300
binsearch: clarify variable name in greater-than-or-equal search
The exact match variable was called "result" following the other
macros, which confused me for a moment.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 3c9483f7f9f4b7b98418df76ccb5000a132a7003
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Wed Jan 11 16:15:42 2017 +1300
replmd: fix variable names in replmd_check_upgrade_links
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 5f52a8fbe78b5e775f7640060964e4ebe69fbcc8
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Thu Dec 29 15:08:00 2016 +1300
replmd: replmd_check_upgrade_links() only checks the first DN
This assumes the links (on an object in the database) are either all in
the old format or all in the new.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 225fa430d1e17b7b8e241c24e2e51a513de00699
Author: Andrew Bartlett <abartlet at samba.org>
Date: Fri Feb 3 14:39:00 2017 +1300
replmd: pass replmd_private down to replmd_add_backlink()
This is not much saving, but we are soon going to need replmd_private
in the intermediate layers (e.g. replmd_modify_la_add).
Pair-programmed-with: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
commit 10e8bc103a2167082a6aa6f47d9511ac4f8b6339
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Fri Dec 23 14:18:13 2016 +1300
replmd: Fix some whitespace in repl_meta_data.c
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 996aafdbff392775716c272301b38958367d7b1e
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Sat Dec 17 22:04:59 2016 +1300
replmd: check whether list is already sorted in get_parsed_dns()
If they are we can avoid the sort.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit a9e0e7a9ef93bde4da412122ab0abcaa34659d4d
Author: Andrew Bartlett <abartlet at samba.org>
Date: Wed Jan 4 21:27:58 2017 +1300
selftest: Do not test for link ordering in tombstones_expunge test
By testing only for the DNs that are returned we do not change the strictness of
the test, because it is a test of the match rule which applies to the whole
object, not the returned values.
However, when this code asserted the returned order of the links, it prevents
us from changing this order. This order was not deterministic across DCs
but as this test ran against an offline DB, it was able to assume a
particular order.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
commit 2fab7585e7f6093f161056ad42e83325ad3d623d
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Wed Feb 1 14:21:22 2017 +1300
s4/linked_attribute tests: remove helper function unused parameter
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit d3dec7a2329ea77101e97cd542e137b6becf7ba1
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Thu Feb 2 13:57:16 2017 +1300
s4/linked_attribute tests: try adding linked attributes directly
Previously we have only added linked attributes using a modify.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Pair-programmed-with: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit b69f48e19be0d3724768716663c4bfb55ab706a3
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Wed Feb 1 14:19:36 2017 +1300
s4/linked_attribute tests: test with the relax control
We had a theory this caused problems. It didn't, but the tests are
still worthwhile.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 6bee3faadd5c07ef81cd158d8c07990f6ee73873
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Thu Jan 12 11:57:17 2017 +1300
s4/linked_attribute tests: compare link lists in sorted order
This isn't functionally different[1] from the previous use of set(),
but it makes the error output easier to read.
[1] OK, it will also show duplicates, which we really don't expect and
would definitely want to see.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 069c4adbc4108dae686f86292aa19563f200f4fb
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Thu Jan 12 11:53:15 2017 +1300
s4/linked_attribute tests: remove unused code
We don't test for sort order because we don't depend on it. So this
test was never used.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 1647f0cedc2c6cecf76513bfacb3bf2813ec5c7a
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Wed Jan 11 12:26:13 2017 +1300
s4/linked_attribute tests: add multiple links and replace tests
Also a "delete all" test.
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 35029286f0ae213e433f31d45d343336fbc683fd
Author: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Date: Wed Jan 11 12:19:21 2017 +1300
s4/linked_attributes test: pep8 tidy-up, remove unused imports
Signed-off-by: Douglas Bagnall <douglas.bagnall at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
-----------------------------------------------------------------------
Summary of changes:
lib/util/binsearch.h | 16 +-
lib/util/tests/binsearch.c | 9 +-
source4/dsdb/samdb/ldb_modules/repl_meta_data.c | 1140 ++++++++++++++------
source4/dsdb/samdb/ldb_modules/samba_dsdb.c | 125 ++-
source4/dsdb/samdb/ldb_modules/vlv_pagination.c | 6 +-
source4/dsdb/samdb/samdb.h | 6 +
source4/dsdb/schema/schema_set.c | 5 +
source4/dsdb/tests/python/linked_attributes.py | 303 +++++-
.../expected-match-rule-links.ldif | 15 -
source4/selftest/tests.py | 5 +
source4/setup/provision_init.ldif | 1 +
source4/setup/tests/blackbox_supported_features.sh | 86 ++
testprogs/blackbox/tombstones-expunge.sh | 2 +-
13 files changed, 1335 insertions(+), 384 deletions(-)
create mode 100755 source4/setup/tests/blackbox_supported_features.sh
Changeset truncated at 500 lines:
diff --git a/lib/util/binsearch.h b/lib/util/binsearch.h
index 001f2c5..4204c9c 100644
--- a/lib/util/binsearch.h
+++ b/lib/util/binsearch.h
@@ -86,31 +86,31 @@
like BINARY_ARRAY_SEARCH_V, but if an exact result is not found, the 'next'
argument will point to the element after the place where the exact result
would have been. If an exact result is found, 'next' will be NULL. If the
- target is beyond the end of the list, both 'result' and 'next' will be NULL.
+ target is beyond the end of the list, both 'exact' and 'next' will be NULL.
Unlike other binsearch macros, where there are several elements that compare
- the same, the result will always point to the first one.
+ the same, the exact result will always point to the first one.
If you don't care to distinguish between the 'greater than' and 'equals'
- cases, you can use the same pointer for both 'result' and 'next'.
+ cases, you can use the same pointer for both 'exact' and 'next'.
As with all the binsearch macros, the comparison function is always called
with the search term first.
*/
#define BINARY_ARRAY_SEARCH_GTE(array, array_size, target, comparison_fn, \
- result, next) do { \
+ exact, next) do { \
int32_t _b, _e; \
- (result) = NULL; (next) = NULL; \
+ (exact) = NULL; (next) = NULL; \
if ((array_size) > 0) { \
for (_b = 0, _e = (array_size)-1; _b <= _e; ) { \
int32_t _i = (_b + _e) / 2; \
- int _r = comparison_fn(target, array[_i]); \
+ int _r = comparison_fn(target, &array[_i]); \
if (_r == 0) { \
- (result) = &array[_i]; \
+ (exact) = &array[_i]; \
_e = _i - 1; \
} else if (_r < 0) { _e = _i - 1; \
} else { _b = _i + 1; } \
} \
- if ((result) == NULL &&_b < (array_size)) { \
+ if ((exact) == NULL &&_b < (array_size)) { \
(next) = &array[_b]; \
} } } while (0)
diff --git a/lib/util/tests/binsearch.c b/lib/util/tests/binsearch.c
index c6ef47e..b3ecda1 100644
--- a/lib/util/tests/binsearch.c
+++ b/lib/util/tests/binsearch.c
@@ -31,6 +31,11 @@ static int int_cmp(int a, int b)
return a - b;
}
+static int int_cmp_p(int a, int *b)
+{
+ return a - *b;
+}
+
static bool test_binsearch_v(struct torture_context *tctx)
{
int array[] = { -11, -7, 0, 1, 723, 1000000};
@@ -72,7 +77,7 @@ static bool test_binsearch_gte(struct torture_context *tctx)
i, target);
BINARY_ARRAY_SEARCH_GTE(array, a_len, target,
- int_cmp, result, next);
+ int_cmp_p, result, next);
if (result == NULL) {
/* we think there is no exact match */
@@ -134,7 +139,7 @@ static bool test_binsearch_gte(struct torture_context *tctx)
i, target);
BINARY_ARRAY_SEARCH_GTE(array, a_len, target,
- int_cmp, result, result);
+ int_cmp_p, result, result);
if (result == NULL) {
/* we think the target is greater than all elements */
diff --git a/source4/dsdb/samdb/ldb_modules/repl_meta_data.c b/source4/dsdb/samdb/ldb_modules/repl_meta_data.c
index 7d12c23..6e041c7 100644
--- a/source4/dsdb/samdb/ldb_modules/repl_meta_data.c
+++ b/source4/dsdb/samdb/ldb_modules/repl_meta_data.c
@@ -69,6 +69,7 @@ struct replmd_private {
} *ncs;
struct ldb_dn *schema_dn;
bool originating_updates;
+ bool sorted_links;
};
struct la_entry {
@@ -236,16 +237,37 @@ static int replmd_init(struct ldb_module *module)
{
struct replmd_private *replmd_private;
struct ldb_context *ldb = ldb_module_get_ctx(module);
-
+ static const char *samba_dsdb_attrs[] = { SAMBA_COMPATIBLE_FEATURES_ATTR, NULL };
+ struct ldb_dn *samba_dsdb_dn;
+ struct ldb_result *res;
+ int ret;
+ TALLOC_CTX *frame = talloc_stackframe();
replmd_private = talloc_zero(module, struct replmd_private);
if (replmd_private == NULL) {
ldb_oom(ldb);
+ TALLOC_FREE(frame);
return LDB_ERR_OPERATIONS_ERROR;
}
ldb_module_set_private(module, replmd_private);
replmd_private->schema_dn = ldb_get_schema_basedn(ldb);
+ samba_dsdb_dn = ldb_dn_new(frame, ldb, "@SAMBA_DSDB");
+ if (!samba_dsdb_dn) {
+ TALLOC_FREE(frame);
+ return ldb_oom(ldb);
+ }
+
+ ret = dsdb_module_search_dn(module, frame, &res, samba_dsdb_dn,
+ samba_dsdb_attrs, DSDB_FLAG_NEXT_MODULE, NULL);
+ if (ret == LDB_SUCCESS) {
+ replmd_private->sorted_links
+ = ldb_msg_check_string_attribute(res->msgs[0],
+ SAMBA_COMPATIBLE_FEATURES_ATTR,
+ SAMBA_SORTED_LINKS_FEATURE);
+ }
+ TALLOC_FREE(frame);
+
return ldb_next_init(module);
}
@@ -410,14 +432,16 @@ static int replmd_process_backlink(struct ldb_module *module, struct la_backlink
add a backlink to the list of backlinks to add/delete in the prepare
commit
*/
-static int replmd_add_backlink(struct ldb_module *module, const struct dsdb_schema *schema,
- struct GUID *forward_guid, struct GUID *target_guid,
- bool active, const struct dsdb_attribute *schema_attr, bool immediate)
+static int replmd_add_backlink(struct ldb_module *module,
+ struct replmd_private *replmd_private,
+ const struct dsdb_schema *schema,
+ struct GUID *forward_guid,
+ struct GUID *target_guid, bool active,
+ const struct dsdb_attribute *schema_attr,
+ bool immediate)
{
const struct dsdb_attribute *target_attr;
struct la_backlink *bl;
- struct replmd_private *replmd_private =
- talloc_get_type_abort(ldb_module_get_private(module), struct replmd_private);
target_attr = dsdb_attribute_by_linkID(schema, schema_attr->linkID ^ 1);
if (!target_attr) {
@@ -838,64 +862,72 @@ static int replmd_build_la_val(TALLOC_CTX *mem_ctx, struct ldb_val *v, struct ds
const struct GUID *invocation_id, uint64_t seq_num,
uint64_t local_usn, NTTIME nttime, uint32_t version, bool deleted);
+struct parsed_dn {
+ struct dsdb_dn *dsdb_dn;
+ struct GUID guid;
+ struct ldb_val *v;
+};
+
+static int get_parsed_dns(struct ldb_module *module, TALLOC_CTX *mem_ctx,
+ struct ldb_message_element *el, struct parsed_dn **pdn,
+ const char *ldap_oid, struct ldb_request *parent);
/*
fix up linked attributes in replmd_add.
This involves setting up the right meta-data in extended DN
components, and creating backlinks to the object
*/
-static int replmd_add_fix_la(struct ldb_module *module, struct ldb_message_element *el,
+static int replmd_add_fix_la(struct ldb_module *module, TALLOC_CTX *mem_ctx,
+ struct replmd_private *replmd_private,
+ struct ldb_message_element *el,
uint64_t seq_num, const struct GUID *invocationId, NTTIME now,
- struct GUID *guid, const struct dsdb_attribute *sa, struct ldb_request *parent)
+ struct GUID *guid, const struct dsdb_attribute *sa,
+ struct ldb_request *parent)
{
unsigned int i;
- TALLOC_CTX *tmp_ctx = talloc_new(el->values);
+ TALLOC_CTX *tmp_ctx = talloc_new(mem_ctx);
struct ldb_context *ldb = ldb_module_get_ctx(module);
-
+ struct parsed_dn *pdn;
/* We will take a reference to the schema in replmd_add_backlink */
const struct dsdb_schema *schema = dsdb_get_schema(ldb, NULL);
+ struct ldb_val *new_values = NULL;
- for (i=0; i<el->num_values; i++) {
- struct ldb_val *v = &el->values[i];
- struct dsdb_dn *dsdb_dn = dsdb_dn_parse(tmp_ctx, ldb, v, sa->syntax->ldap_oid);
- struct GUID target_guid;
- NTSTATUS status;
- int ret;
-
- if (dsdb_dn == NULL) {
- talloc_free(tmp_ctx);
- return LDB_ERR_INVALID_DN_SYNTAX;
- }
+ int ret = get_parsed_dns(module, tmp_ctx, el, &pdn,
+ sa->syntax->ldap_oid, parent);
+ if (ret != LDB_SUCCESS) {
+ talloc_free(tmp_ctx);
+ return ret;
+ }
- /* note that the DN already has the extended
- components from the extended_dn_store module */
- status = dsdb_get_extended_dn_guid(dsdb_dn->dn, &target_guid, "GUID");
- if (!NT_STATUS_IS_OK(status) || GUID_all_zero(&target_guid)) {
- ret = dsdb_module_guid_by_dn(module, dsdb_dn->dn, &target_guid, parent);
- if (ret != LDB_SUCCESS) {
- talloc_free(tmp_ctx);
- return ret;
- }
- ret = dsdb_set_extended_dn_guid(dsdb_dn->dn, &target_guid, "GUID");
- if (ret != LDB_SUCCESS) {
- talloc_free(tmp_ctx);
- return ret;
- }
- }
+ new_values = talloc_array(tmp_ctx, struct ldb_val, el->num_values);
+ if (new_values == NULL) {
+ ldb_module_oom(module);
+ talloc_free(tmp_ctx);
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
- ret = replmd_build_la_val(el->values, v, dsdb_dn, invocationId,
+ for (i = 0; i < el->num_values; i++) {
+ struct parsed_dn *p = &pdn[i];
+ ret = replmd_build_la_val(el->values, p->v, p->dsdb_dn,
+ invocationId,
seq_num, seq_num, now, 0, false);
if (ret != LDB_SUCCESS) {
talloc_free(tmp_ctx);
return ret;
}
- ret = replmd_add_backlink(module, schema, guid, &target_guid, true, sa, false);
+ /* This is the only place we are doing deferred back-links */
+ ret = replmd_add_backlink(module, replmd_private,
+ schema, guid, &p->guid, true, sa,
+ false);
if (ret != LDB_SUCCESS) {
talloc_free(tmp_ctx);
return ret;
}
+
+ new_values[i] = *p->v;
}
+ el->values = talloc_steal(mem_ctx, new_values);
talloc_free(tmp_ctx);
return LDB_SUCCESS;
@@ -1072,7 +1104,9 @@ static int replmd_add(struct ldb_module *module, struct ldb_request *req)
}
if (sa->linkID != 0 && functional_level > DS_DOMAIN_FUNCTION_2000) {
- ret = replmd_add_fix_la(module, e, ac->seq_num,
+ ret = replmd_add_fix_la(module, msg->elements,
+ replmd_private, e,
+ ac->seq_num,
&ac->our_invocation_id, now,
&guid, sa, req);
if (ret != LDB_SUCCESS) {
@@ -1522,7 +1556,7 @@ static int replmd_update_rpmd(struct ldb_module *module,
bool rmd_is_provided;
bool rmd_is_just_resorted = false;
const char *not_rename_attrs[4 + msg->num_elements];
-
+
if (rename_attrs) {
attrs = rename_attrs;
} else {
@@ -1781,41 +1815,182 @@ static int replmd_update_rpmd(struct ldb_module *module,
return LDB_SUCCESS;
}
-struct parsed_dn {
- struct dsdb_dn *dsdb_dn;
- struct GUID guid;
- struct ldb_val *v;
+struct compare_ctx {
+ struct GUID *guid;
+ struct ldb_context *ldb;
+ TALLOC_CTX *mem_ctx;
+ const char *ldap_oid;
+ int err;
+ const struct GUID *invocation_id;
};
+/* When a parsed_dn comes from the database, sometimes it is not really parsed. */
+
+static int really_parse_trusted_dn(TALLOC_CTX *mem_ctx, struct ldb_context *ldb,
+ struct parsed_dn *pdn, const char *ldap_oid)
+{
+ NTSTATUS status;
+ struct dsdb_dn *dsdb_dn = dsdb_dn_parse_trusted(mem_ctx, ldb, pdn->v,
+ ldap_oid);
+ if (dsdb_dn == NULL) {
+ return LDB_ERR_INVALID_DN_SYNTAX;
+ }
+
+ status = dsdb_get_extended_dn_guid(dsdb_dn->dn, &pdn->guid, "GUID");
+ if (!NT_STATUS_IS_OK(status)) {
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
+ pdn->dsdb_dn = dsdb_dn;
+ return LDB_SUCCESS;
+}
+
+/*
+ * We choose, as the sort order, the same order as is used in DRS replication,
+ * which is the memcmp() order of the NDR GUID, not that obtained from
+ * GUID_compare().
+ *
+ * This means that sorted links will be in the same order as a new DC would
+ * see them.
+ */
+static int ndr_guid_compare(struct GUID *guid1, struct GUID *guid2)
+{
+ uint8_t v1_data[16];
+ struct ldb_val v1 = data_blob_const(v1_data, sizeof(v1_data));
+ uint8_t v2_data[16];
+ struct ldb_val v2 = data_blob_const(v2_data, sizeof(v2_data));
+
+ /* This can't fail */
+ ndr_push_struct_into_fixed_blob(&v1, guid1,
+ (ndr_push_flags_fn_t)ndr_push_GUID);
+ /* This can't fail */
+ ndr_push_struct_into_fixed_blob(&v2, guid2,
+ (ndr_push_flags_fn_t)ndr_push_GUID);
+ return data_blob_cmp(&v1, &v2);
+}
+
static int parsed_dn_compare(struct parsed_dn *pdn1, struct parsed_dn *pdn2)
{
- return GUID_compare(&pdn1->guid, &pdn2->guid);
+ return ndr_guid_compare(&pdn1->guid, &pdn2->guid);
}
-static int GUID_compare_struct(struct GUID *g1, struct GUID g2)
+static int la_guid_compare_with_trusted_dn(struct compare_ctx *ctx,
+ struct parsed_dn *p)
{
- return GUID_compare(g1, &g2);
+ /*
+ * This works like a standard compare function in its return values,
+ * but has an extra trick to deal with errors: zero is returned and
+ * ctx->err is set to the ldb error code.
+ *
+ * That is, if (as is expected in most cases) you get a non-zero
+ * result, you don't need to check for errors.
+ *
+ * We assume the second argument refers to a DN is from the database
+ * and has a GUID -- but this GUID might not have been parsed out yet.
+ */
+ if (p->dsdb_dn == NULL) {
+ int ret = really_parse_trusted_dn(ctx->mem_ctx, ctx->ldb, p,
+ ctx->ldap_oid);
+ if (ret != LDB_SUCCESS) {
+ ctx->err = ret;
+ return 0;
+ }
+ }
+ return ndr_guid_compare(ctx->guid, &p->guid);
}
-static struct parsed_dn *parsed_dn_find(struct parsed_dn *pdn,
- unsigned int count, struct GUID *guid,
- struct ldb_dn *dn)
+
+
+static int parsed_dn_find(struct ldb_context *ldb, struct parsed_dn *pdn,
+ unsigned int count,
+ struct GUID *guid,
+ struct ldb_dn *target_dn,
+ struct parsed_dn **exact,
+ struct parsed_dn **next,
+ const char *ldap_oid)
{
- struct parsed_dn *ret;
unsigned int i;
- if (dn && GUID_all_zero(guid)) {
- /* when updating a link using DRS, we sometimes get a
- NULL GUID. We then need to try and match by DN */
- for (i=0; i<count; i++) {
- if (ldb_dn_compare(pdn[i].dsdb_dn->dn, dn) == 0) {
- dsdb_get_extended_dn_guid(pdn[i].dsdb_dn->dn, guid, "GUID");
- return &pdn[i];
+ struct compare_ctx ctx;
+ if (pdn == NULL) {
+ *exact = NULL;
+ *next = NULL;
+ return LDB_SUCCESS;
+ }
+
+ if (unlikely(GUID_all_zero(guid))) {
+ /*
+ * When updating a link using DRS, we sometimes get a NULL
+ * GUID when a forward link has been deleted and its GUID has
+ * for some reason been forgotten. The best we can do is try
+ * and match by DN via a linear search. Note that this
+ * probably only happens in the ADD case, in which we only
+ * allow modification of link if it is already deleted, so
+ * this seems very close to an elaborate NO-OP, but we are not
+ * quite prepared to declare it so.
+ *
+ * If the DN is not in our list, we have to add it to the
+ * beginning of the list, where it would naturally sort.
+ */
+ struct parsed_dn *p;
+ if (target_dn == NULL) {
+ /* We don't know the target DN, so we can't search for DN */
+ DEBUG(1, ("parsed_dn_find has a NULL GUID for a linked "
+ "attribute but we don't have a DN to compare "
+ "it with\n"));
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
+ *exact = NULL;
+ *next = NULL;
+
+ DEBUG(3, ("parsed_dn_find has a NULL GUID for a link to DN "
+ "%s; searching through links for it",
+ ldb_dn_get_linearized(target_dn)));
+
+ for (i = 0; i < count; i++) {
+ int cmp;
+ p = &pdn[i];
+ if (p->dsdb_dn == NULL) {
+ int ret = really_parse_trusted_dn(pdn, ldb, p, ldap_oid);
+ if (ret != LDB_SUCCESS) {
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
+ }
+
+ cmp = ldb_dn_compare(p->dsdb_dn->dn, target_dn);
+ if (cmp == 0) {
+ *exact = p;
+ return LDB_SUCCESS;
}
}
- return NULL;
+ /*
+ * Here we have a null guid which doesn't match any existing
+ * link. This is a bit unexpected because null guids occur
+ * when a forward link has been deleted and we are replicating
+ * that deletion.
+ *
+ * The best thing to do is weep into the logs and add the
+ * offending link to the beginning of the list which is
+ * at least the correct sort position.
+ */
+ DEBUG(1, ("parsed_dn_find has been given a NULL GUID for a "
+ "link to unknown DN %s\n",
+ ldb_dn_get_linearized(target_dn)));
+ *next = pdn;
+ return LDB_SUCCESS;
}
- BINARY_ARRAY_SEARCH(pdn, count, guid, guid, GUID_compare_struct, ret);
- return ret;
+
+ ctx.guid = guid;
+ ctx.ldb = ldb;
+ ctx.mem_ctx = pdn;
+ ctx.ldap_oid = ldap_oid;
+ ctx.err = 0;
+
+ BINARY_ARRAY_SEARCH_GTE(pdn, count, &ctx, la_guid_compare_with_trusted_dn,
+ *exact, *next);
+
+ if (ctx.err != 0) {
+ return ctx.err;
+ }
+ return LDB_SUCCESS;
}
/*
@@ -1827,6 +2002,7 @@ static int get_parsed_dns(struct ldb_module *module, TALLOC_CTX *mem_ctx,
const char *ldap_oid, struct ldb_request *parent)
{
unsigned int i;
+ bool values_are_sorted = true;
struct ldb_context *ldb = ldb_module_get_ctx(module);
if (el == NULL) {
@@ -1856,7 +2032,8 @@ static int get_parsed_dns(struct ldb_module *module, TALLOC_CTX *mem_ctx,
dn = p->dsdb_dn->dn;
status = dsdb_get_extended_dn_guid(dn, &p->guid, "GUID");
- if (NT_STATUS_EQUAL(status, NT_STATUS_OBJECT_NAME_NOT_FOUND)) {
+ if (NT_STATUS_EQUAL(status, NT_STATUS_OBJECT_NAME_NOT_FOUND) ||
+ unlikely(GUID_all_zero(&p->guid))) {
/* we got a DN without a GUID - go find the GUID */
int ret = dsdb_module_guid_by_dn(module, dn, &p->guid, parent);
if (ret != LDB_SUCCESS) {
@@ -1876,12 +2053,60 @@ static int get_parsed_dns(struct ldb_module *module, TALLOC_CTX *mem_ctx,
} else if (!NT_STATUS_IS_OK(status)) {
--
Samba Shared Repository
More information about the samba-cvs
mailing list