[SCM] Samba Shared Repository - branch master updated
Andrew Bartlett
abartlet at samba.org
Mon Apr 8 03:10:03 UTC 2019
The branch, master has been updated
via 21d501bfa8f selftest: Correct name of flapping smb2.notify test
via 15d1ecdca6a ldb: version 2.0.0
via 9d0b0036239 ldb_mdb: Add some warnings about poorly constructed callbacks
via 2e05fd785a3 ldb: tests for <= and >= integer indexing with duplicates
via 18438c8af2f ldb: tests for <= and >= integer indexing
via 6062d7241de ldb: Add ORDERED_INTEGER to the proto-schema handling
via db584d50817 schema_syntax: Add comments for our index format functions
via c9b2a37268e ldb: activating <= and >= indexing for integers
via f7756065160 ldb_kv_index: Add a giant comment in regards to index_format_fn
via a894515229f ldb_kv_index: Make the edge keys slightly cleaner and generic
via 9b3021b8d6b ldb: <= and >= indexed searching
via 1b5df443314 lmdb: iterate_range cmocka testing
via e4ea408f52e lmdb: iterate_range implementation
via fc2e521a5e3 libnet vampire: NULL access bug fix
from 43cacaad571 ctdb: Fix a typo
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 21d501bfa8f66d98d2b5dfd3ffa2a9dd07af5dbd
Author: Andrew Bartlett <abartlet at samba.org>
Date: Mon Apr 8 14:04:08 2019 +1200
selftest: Correct name of flapping smb2.notify test
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Mon Apr 8 03:09:42 UTC 2019 on sn-devel-144
commit 15d1ecdca6a4fbceddefd7ef4b8a9b912c067207
Author: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Wed Mar 20 13:52:16 2019 +1300
ldb: version 2.0.0
* Version bump for adding index_format_fn to the schema syntax structure.
* Range index support added, allowing <= and >= operations to be indexed
* Improved reindex performance by setting the in-memory TDB hash size correctly
Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 9d0b00362398f5a14e735dc79a002c344b47fe8c
Author: Garming Sam <garming at catalyst.net.nz>
Date: Thu Apr 4 12:04:47 2019 +1300
ldb_mdb: Add some warnings about poorly constructed callbacks
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 2e05fd785a3b7355abb6fea3003f244a0a958581
Author: Garming Sam <garming at catalyst.net.nz>
Date: Wed Apr 3 16:12:35 2019 +1300
ldb: tests for <= and >= integer indexing with duplicates
We need to make sure that duplicates are correctly returned (uSNChanged
for instance is UNIQUE but, we should be able to index on attributes
which are not unique).
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 18438c8af2fb64ea3580cee92b4744a759a0ded8
Author: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Mon Mar 11 16:39:13 2019 +1300
ldb: tests for <= and >= integer indexing
Testing max, min and negative values for indexed 32 and 64 bit types.
This has to be done in two different files because the 64 bit type is
LDB_SYNTAX_INTEGER which is implemented at the ldb level, while the 32
bit is added in the ldb-samba module. Schema syntax binding added for
ldb-samba.
We also need to make sure that full scans are not invoked for LMDB.
Pair-programmed-with: Garming Sam <garming at catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 6062d7241deb931c35700dd0449ac7e34e1d5b2a
Author: Garming Sam <garming at catalyst.net.nz>
Date: Mon Mar 25 13:52:42 2019 +1300
ldb: Add ORDERED_INTEGER to the proto-schema handling
Adding ordered integer proto schema handling in kv index cache. This
allows ordered 64 bit integers to be used in cached fields like
@ATTRIBUTES
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit db584d50817b6d0dc186c0b00de47e4c2b6681cf
Author: Garming Sam <garming at catalyst.net.nz>
Date: Fri Apr 5 10:22:28 2019 +1300
schema_syntax: Add comments for our index format functions
We had to devise our own scheme for writing integers in a human readable
format which also sorted correctly numerically. This might look a bit
confusing to outsiders, so here's a large comment as a peace offering.
Pair-programmed-with: Tim Beale <timbeale at catalyst.net.nz>
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit c9b2a37268e793f2d8a8e3a6a340d2bc869f3f0e
Author: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Thu Mar 14 18:05:23 2019 +1300
ldb: activating <= and >= indexing for integers
Activating <= and >= mdb indexing in samba for int32 and int64 attributes by:
1. Adding index_format_fn to LDB_SYNTAX_SAMBA_INT32 in ldb_samba
2. Cloning the 64bit LDB_SYNTAX_INTEGER type as LDB_SYNTAX_ORDERED_INTEGER
3. Adding index_format_fn to the new type
4. Modifying LargeInteger use the new type in samba schema
5. Bumping the index version to trigger reindexing
Pair-programmed-with: Garming Sam <garming at catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit f775606516015715379dd76fa545da7c02c65e45
Author: Garming Sam <garming at catalyst.net.nz>
Date: Thu Apr 4 15:02:15 2019 +1300
ldb_kv_index: Add a giant comment in regards to index_format_fn
The reason we needed it in the first place was that the original
canonicalize is being used for non-index functions and it never produced
the right order originally (at least for integers).
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit a894515229f1edc3f07ea08faafdbfba16cf32ff
Author: Garming Sam <garming at catalyst.net.nz>
Date: Thu Apr 4 14:36:08 2019 +1300
ldb_kv_index: Make the edge keys slightly cleaner and generic
It makes no difference in our standard case because \0 will always go
before any value for our index_format_fn, but this is better for
correctness (in case we do mess up our NUL terminations elsewhere).
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 9b3021b8d6b963c5023cc16b9f764707fdbcafbc
Author: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Mon Mar 4 19:06:31 2019 +1300
ldb: <= and >= indexed searching
Full implementation of <= and >= indexed searching using iterate_range
backend operation. Adds index_format_fn to ldb_schema_syntax so
requires an ABI version bump. The function must be provided for any
type for which <= and >= indexing is required, and must return a
lexicographically ordered canonicalization of a value. This causes
index entries to be written in correct order to the database, so
iterate_range on the index DNs can be used.
ldb_kv_index_key is modified to return an index DN with attribute name
but without value if an empty value is provided. This is needed for
constructing keys that match the beginning or end of an index DN range.
Pair-programmed-with: Garming Sam <garming at catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 1b5df44331434256228070df35217c837a8cd838
Author: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Date: Mon Mar 4 15:08:25 2019 +1300
lmdb: iterate_range cmocka testing
Cmocka testing for LMDB iterate_range operation added in previous commit.
Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit e4ea408f52eeb958f646dfa4ef304357a080bdd1
Author: Garming Sam <garming at catalyst.net.nz>
Date: Mon Mar 4 12:50:24 2019 +1300
lmdb: iterate_range implementation
Adding iterate_range to LDB API and implementing in LMDB. This
operation takes a start_key and end_key and returns all records between
the two, inclusive of both. This will be used to implementing indexing
for <= and >= expressions.
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit fc2e521a5e34988547de27a024a180b83d6504ee
Author: Garming Sam <garming at catalyst.net.nz>
Date: Tue Mar 12 11:16:38 2019 +1300
libnet vampire: NULL access bug fix
NULL pointer access bug fix
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Signed-off-by: Aaron Haslett <aaronhaslett at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
-----------------------------------------------------------------------
Summary of changes:
lib/ldb-samba/ldif_handlers.c | 64 ++++
lib/ldb-samba/pyldb.c | 31 ++
lib/ldb-samba/tests/index.py | 192 +++++++++++
lib/ldb/ABI/{ldb-1.5.1.sigs => ldb-2.0.0.sigs} | 0
...yldb-util-1.1.10.sigs => pyldb-util-2.0.0.sigs} | 0
...-util-1.1.10.sigs => pyldb-util.py3-2.0.0.sigs} | 0
lib/ldb/common/attrib_handlers.c | 117 ++++++-
lib/ldb/include/ldb.h | 10 +
lib/ldb/ldb_key_value/ldb_kv.h | 5 +
lib/ldb/ldb_key_value/ldb_kv_cache.c | 4 +
lib/ldb/ldb_key_value/ldb_kv_index.c | 370 ++++++++++++++++++++-
lib/ldb/ldb_mdb/ldb_mdb.c | 144 ++++++++
lib/ldb/ldb_tdb/ldb_tdb.c | 17 +
lib/ldb/pyldb.c | 1 +
lib/ldb/tests/ldb_kv_ops_test.c | 127 ++++++-
lib/ldb/tests/python/index.py | 122 +++++++
lib/ldb/wscript | 2 +-
python/samba/tests/complex_expressions.py | 15 +
selftest/flapping.d/smb2_notify | 2 +-
source4/dsdb/schema/schema_set.c | 4 +-
source4/dsdb/schema/schema_syntax.c | 2 +-
source4/libnet/libnet_vampire.c | 7 +-
source4/selftest/tests.py | 1 +
23 files changed, 1213 insertions(+), 24 deletions(-)
create mode 100644 lib/ldb-samba/tests/index.py
copy lib/ldb/ABI/{ldb-1.5.1.sigs => ldb-2.0.0.sigs} (100%)
copy lib/ldb/ABI/{pyldb-util-1.1.10.sigs => pyldb-util-2.0.0.sigs} (100%)
copy lib/ldb/ABI/{pyldb-util-1.1.10.sigs => pyldb-util.py3-2.0.0.sigs} (100%)
Changeset truncated at 500 lines:
diff --git a/lib/ldb-samba/ldif_handlers.c b/lib/ldb-samba/ldif_handlers.c
index d38cdd0c9a3..23d0860dd9b 100644
--- a/lib/ldb-samba/ldif_handlers.c
+++ b/lib/ldb-samba/ldif_handlers.c
@@ -835,6 +835,69 @@ static int ldif_canonicalise_int32(struct ldb_context *ldb, void *mem_ctx,
return 0;
}
+/*
+ * Lexicographically sorted representation for a 32-bit integer
+ *
+ * [ INT32_MIN ... -3, -2, -1 | 0 | +1, +2, +3 ... INT32_MAX ]
+ * n o p
+ *
+ * Refer to the comment in lib/ldb/common/attrib_handlers.c for the
+ * corresponding documentation for 64-bit integers.
+ *
+ * The same rules apply but use INT32_MIN and INT32_MAX.
+ *
+ * String representation padding is done to 10 characters.
+ *
+ * INT32_MAX = 2^31 - 1 = 2147483647 (10 characters long)
+ *
+ */
+static int ldif_index_format_int32(struct ldb_context *ldb,
+ void *mem_ctx,
+ const struct ldb_val *in,
+ struct ldb_val *out)
+{
+ int32_t i;
+ int ret;
+ char prefix;
+ size_t len;
+
+ ret = val_to_int32(in, &i);
+ if (ret != LDB_SUCCESS) {
+ return ret;
+ }
+
+ if (i < 0) {
+ /*
+ * i is negative, so this is subtraction rather than
+ * wrap-around.
+ */
+ prefix = 'n';
+ i = INT32_MAX + i + 1;
+ } else if (i > 0) {
+ prefix = 'p';
+ } else {
+ prefix = 'o';
+ }
+
+ out->data = (uint8_t *) talloc_asprintf(mem_ctx, "%c%010ld", prefix, (long)i);
+ if (out->data == NULL) {
+ ldb_oom(ldb);
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
+
+ len = talloc_array_length(out->data) - 1;
+ if (len != 11) {
+ ldb_debug(ldb, LDB_DEBUG_ERROR,
+ __location__ ": expected index format str %s to"
+ " have length 11 but got %zu",
+ (char*)out->data, len);
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
+
+ out->length = 11;
+ return 0;
+}
+
/* Comparison of two 32-bit integers */
static int ldif_comparison_int32(struct ldb_context *ldb, void *mem_ctx,
const struct ldb_val *v1, const struct ldb_val *v2)
@@ -1427,6 +1490,7 @@ static const struct ldb_schema_syntax samba_syntaxes[] = {
.ldif_read_fn = ldb_handler_copy,
.ldif_write_fn = ldb_handler_copy,
.canonicalise_fn = ldif_canonicalise_int32,
+ .index_format_fn = ldif_index_format_int32,
.comparison_fn = ldif_comparison_int32,
.operator_fn = samba_syntax_operator_fn
},{
diff --git a/lib/ldb-samba/pyldb.c b/lib/ldb-samba/pyldb.c
index 0cf550bc7a5..397e11062b6 100644
--- a/lib/ldb-samba/pyldb.c
+++ b/lib/ldb-samba/pyldb.c
@@ -30,6 +30,7 @@
#include "lib/ldb-samba/ldif_handlers.h"
#include "auth/pyauth.h"
#include "source4/dsdb/common/util.h"
+#include "lib/ldb/include/ldb_private.h"
static PyObject *pyldb_module;
@@ -202,6 +203,29 @@ static PyObject *py_ldb_set_session_info(PyObject *self, PyObject *args)
Py_RETURN_NONE;
}
+static PyObject *py_ldb_samba_schema_attribute_add(PyLdbObject *self,
+ PyObject *args)
+{
+ char *attribute, *syntax;
+ const struct ldb_schema_syntax *s;
+ unsigned int flags;
+ int ret;
+ struct ldb_context *ldb_ctx;
+
+ if (!PyArg_ParseTuple(args, "sIs", &attribute, &flags, &syntax))
+ return NULL;
+
+ ldb_ctx = pyldb_Ldb_AsLdbContext(self);
+
+ s = ldb_samba_syntax_by_name(ldb_ctx, syntax);
+ ret = ldb_schema_attribute_add_with_syntax(ldb_ctx, attribute,
+ flags, s);
+
+ PyErr_LDB_ERROR_IS_ERR_RAISE(py_ldb_error, ret, ldb_ctx);
+
+ Py_RETURN_NONE;
+}
+
static PyObject *py_ldb_register_samba_handlers(PyObject *self)
{
struct ldb_context *ldb;
@@ -237,6 +261,9 @@ static PyMethodDef py_samba_ldb_methods[] = {
{ "set_session_info", (PyCFunction)py_ldb_set_session_info, METH_VARARGS,
"set_session_info(session_info)\n"
"Set session info to use when connecting." },
+ { "samba_schema_attribute_add",
+ (PyCFunction)py_ldb_samba_schema_attribute_add,
+ METH_VARARGS, NULL },
{ NULL },
};
@@ -255,6 +282,7 @@ static PyTypeObject PySambaLdb = {
.tp_flags = Py_TPFLAGS_DEFAULT|Py_TPFLAGS_BASETYPE,
};
+
MODULE_INIT_FUNC(_ldb)
{
PyObject *m;
@@ -283,5 +311,8 @@ MODULE_INIT_FUNC(_ldb)
Py_INCREF(&PySambaLdb);
PyModule_AddObject(m, "Ldb", (PyObject *)&PySambaLdb);
+#define ADD_LDB_STRING(val) PyModule_AddStringConstant(m, #val, LDB_## val)
+ ADD_LDB_STRING(SYNTAX_SAMBA_INT32);
+
return m;
}
diff --git a/lib/ldb-samba/tests/index.py b/lib/ldb-samba/tests/index.py
new file mode 100644
index 00000000000..2256e3eceaa
--- /dev/null
+++ b/lib/ldb-samba/tests/index.py
@@ -0,0 +1,192 @@
+#!/usr/bin/env python3
+#
+# Tests for comparison expressions on indexed keys
+#
+# Copyright (C) Andrew Bartlett <abartlet at samba.org> 2019
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+"""Tests for expressions containing comparisons on indexed attributes.
+ Copied from ldb's index.py"""
+
+import os
+from unittest import TestCase
+import sys
+from samba import _ldb
+import shutil
+from ldb import SCOPE_SUBTREE
+from samba.tests.subunitrun import TestProgram
+
+PY3 = sys.version_info > (3, 0)
+
+TDB_PREFIX = "tdb://"
+MDB_PREFIX = "mdb://"
+
+def tempdir():
+ import tempfile
+ try:
+ dir_prefix = os.path.join(os.environ["SELFTEST_PREFIX"], "tmp")
+ except KeyError:
+ dir_prefix = None
+ return tempfile.mkdtemp(dir=dir_prefix)
+
+class LdbBaseTest(TestCase):
+ def setUp(self):
+ super(LdbBaseTest, self).setUp()
+ try:
+ if self.prefix is None:
+ self.prefix = TDB_PREFIX
+ except AttributeError:
+ self.prefix = TDB_PREFIX
+
+ def tearDown(self):
+ super(LdbBaseTest, self).tearDown()
+
+ def url(self):
+ return self.prefix + self.filename
+
+ def flags(self):
+ if self.prefix == MDB_PREFIX:
+ return ldb.FLG_NOSYNC
+ else:
+ return 0
+
+ def options(self):
+ if self.prefix == MDB_PREFIX:
+ return ['disable_full_db_scan_for_self_test:1']
+ else:
+ return None
+
+class LdbTDBIndexedComparisonExpressions(LdbBaseTest):
+ def tearDown(self):
+ shutil.rmtree(self.testdir)
+ super(LdbTDBIndexedComparisonExpressions, self).tearDown()
+
+ # Ensure the LDB is closed now, so we close the FD
+ del(self.l)
+
+ def setUp(self):
+ super(LdbTDBIndexedComparisonExpressions, self).setUp()
+ self.testdir = tempdir()
+ self.filename = os.path.join(self.testdir, "indexedcomptest.ldb")
+ # Note that the maximum key length is set to 54
+ # This accounts for the 4 bytes added by the dn formatting
+ # a leading dn=, and a trailing zero terminator
+ #
+ self.l = _ldb.Ldb(self.url(), options=self.options())
+ self.l.add({"dn": "@ATTRIBUTES"})
+ self.l.add({"dn": "@INDEXLIST",
+ "@IDXATTR": [b"int32attr"],
+ "@IDXONE": [b"1"],
+ "@IDXGUID": [b"objectUUID"],
+ "@IDX_DN_GUID": [b"GUID"]})
+
+ def test_comparison_expression(self):
+ self.l.samba_schema_attribute_add("int32attr", 0,
+ _ldb.SYNTAX_SAMBA_INT32)
+
+ int32_max = 2**31-1
+ int32_min = -2**31
+ test_nums = list(range(-5, 5))
+ test_nums += list(range(int32_max-5, int32_max+1))
+ test_nums += list(range(int32_min, int32_min+5))
+ test_nums = sorted(test_nums)
+
+ for i in test_nums:
+ ouuid = 0x0123456789abcdef + i
+ ouuid_s = bytes(('0' + hex(ouuid)[2:]).encode())
+ self.l.add({"dn": "OU=COMPTESTOU{},DC=SAMBA,DC=ORG".format(i),
+ "objectUUID": ouuid_s,
+ "int32attr": str(i)})
+
+ def assert_int32_expr(expr, py_expr=None):
+ res = self.l.search(base="DC=SAMBA,DC=ORG",
+ scope=SCOPE_SUBTREE,
+ expression="(int32attr%s)" % (expr))
+
+ if not py_expr:
+ py_expr = expr
+ expect = [n for n in test_nums if eval(str(n) + py_expr)]
+ vals = sorted([int(r.get("int32attr")[0]) for r in res])
+ self.assertEqual(len(res), len(expect))
+ self.assertEqual(set(vals), set(expect))
+ self.assertEqual(expect, vals)
+
+ assert_int32_expr(">=-2")
+ assert_int32_expr("<=2")
+ assert_int32_expr(">=" + str(int32_min))
+ assert_int32_expr("<=" + str(int32_min))
+ assert_int32_expr("<=" + str(int32_min+1))
+ assert_int32_expr("<=" + str(int32_max))
+ assert_int32_expr(">=" + str(int32_max))
+ assert_int32_expr(">=" + str(int32_max-1))
+ assert_int32_expr("=10", "==10")
+
+ def test_comparison_expression_duplicates(self):
+ self.l.samba_schema_attribute_add("int32attr", 0,
+ _ldb.SYNTAX_SAMBA_INT32)
+
+ int32_max = 2**31-1
+ int32_min = -2**31
+
+ test_nums = list(range(-5, 5)) * 3
+ test_nums += list(range(-20, 20, 5)) * 2
+ test_nums += list(range(-50, 50, 15))
+ test_nums = sorted(test_nums)
+
+ for i, n in enumerate(test_nums):
+ ouuid = 0x0123456789abcdef + i
+ ouuid_s = bytes(('0' + hex(ouuid)[2:]).encode())
+ self.l.add({"dn": "OU=COMPTESTOU{},DC=SAMBA,DC=ORG".format(i),
+ "objectUUID": ouuid_s,
+ "int32attr": str(n)})
+
+ def assert_int32_expr(expr, py_expr=None):
+ res = self.l.search(base="DC=SAMBA,DC=ORG",
+ scope=SCOPE_SUBTREE,
+ expression="(int32attr%s)" % (expr))
+
+ if not py_expr:
+ py_expr = expr
+ expect = [n for n in test_nums if eval(str(n) + py_expr)]
+ vals = sorted([int(r.get("int32attr")[0]) for r in res])
+ self.assertEqual(len(res), len(expect))
+ self.assertEqual(set(vals), set(expect))
+ self.assertEqual(expect, vals)
+
+ assert_int32_expr(">=-2")
+ assert_int32_expr("<=2")
+ assert_int32_expr(">=" + str(int32_min))
+ assert_int32_expr("<=" + str(int32_min))
+ assert_int32_expr("<=" + str(int32_min+1))
+ assert_int32_expr("<=" + str(int32_max))
+ assert_int32_expr(">=" + str(int32_max))
+ assert_int32_expr(">=" + str(int32_max-1))
+ assert_int32_expr("=-5", "==-5")
+ assert_int32_expr("=5", "==5")
+
+# Run the same tests against an lmdb backend
+class LdbLMDBIndexedComparisonExpressions(LdbTDBIndexedComparisonExpressions):
+
+ def setUp(self):
+ if os.environ.get('HAVE_LMDB', '1') == '0':
+ self.skipTest("No lmdb backend")
+ self.prefix = MDB_PREFIX
+ super(LdbLMDBIndexedComparisonExpressions, self).setUp()
+
+ def tearDown(self):
+ super(LdbLMDBIndexedComparisonExpressions, self).tearDown()
+
+
+TestProgram(module=__name__, opts=[])
diff --git a/lib/ldb/ABI/ldb-1.5.1.sigs b/lib/ldb/ABI/ldb-2.0.0.sigs
similarity index 100%
copy from lib/ldb/ABI/ldb-1.5.1.sigs
copy to lib/ldb/ABI/ldb-2.0.0.sigs
diff --git a/lib/ldb/ABI/pyldb-util-1.1.10.sigs b/lib/ldb/ABI/pyldb-util-2.0.0.sigs
similarity index 100%
copy from lib/ldb/ABI/pyldb-util-1.1.10.sigs
copy to lib/ldb/ABI/pyldb-util-2.0.0.sigs
diff --git a/lib/ldb/ABI/pyldb-util-1.1.10.sigs b/lib/ldb/ABI/pyldb-util.py3-2.0.0.sigs
similarity index 100%
copy from lib/ldb/ABI/pyldb-util-1.1.10.sigs
copy to lib/ldb/ABI/pyldb-util.py3-2.0.0.sigs
diff --git a/lib/ldb/common/attrib_handlers.c b/lib/ldb/common/attrib_handlers.c
index 4b94d392cc6..b5212b73159 100644
--- a/lib/ldb/common/attrib_handlers.c
+++ b/lib/ldb/common/attrib_handlers.c
@@ -146,6 +146,113 @@ static int ldb_canonicalise_Integer(struct ldb_context *ldb, void *mem_ctx,
return 0;
}
+/*
+ * Lexicographically ordered format for a ldap Integer
+ *
+ * [ INT64_MIN ... -3, -2, -1 | 0 | +1, +2, +3 ... INT64_MAX ]
+ * n o p
+ *
+ * For human readability sake, we continue to format the key as a string
+ * (like the canonicalize) rather than store as a fixed binary representation.
+ *
+ * In order to sort the integers in the correct string order, there are three
+ * techniques we use:
+ *
+ * 1. Zero padding
+ * 2. Negative integer inversion
+ * 3. 1-byte prefixes: 'n' < 'o' < 'p'
+ *
+ * 1. To have a fixed-width representation so that 10 sorts after 2 rather than
+ * after 1, we zero pad, like this 4-byte width example:
+ *
+ * 0001, 0002, 0010
+ *
+ * INT64_MAX = 2^63 - 1 = 9223372036854775807 (19 characters long)
+ *
+ * Meaning we need to pad to 19 characters.
+ *
+ * 2. This works for positive integers, but negative integers will still be
+ * sorted backwards, for example:
+ *
+ * -9223372036854775808 ..., -0000000000000000002, -0000000000000000001
+ * INT64_MIN -2 -1
+ *
+ * gets sorted based on string as:
+ *
+ * -0000000000000000001, -0000000000000000002, ... -9223372036854775808
+ *
+ * In order to fix this, we invert the negative integer range, so that they
+ * get sorted the same way as positive numbers. INT64_MIN becomes the lowest
+ * possible non-negative number (zero), and -1 becomes the highest (INT64_MAX).
+ *
+ * The actual conversion applied to negative number 'x' is:
+ * INT64_MAX - abs(x) + 1
+ * (The +1 is needed because abs(INT64_MIN) is one greater than INT64_MAX)
+ *
+ * 3. Finally, we now have two different numbers that map to the same key, e.g.
+ * INT64_MIN maps to -0000000000000000000 and zero maps to 0000000000000000000.
+ * In order to avoid confusion, we give every number a prefix representing its
+ * sign: 'n' for negative numbers, 'o' for zero, and 'p' for positive. (Note
+ * that '+' and '-' weren't used because they sort the wrong way).
+ *
+ * The result is a range of key values that look like this:
+ *
+ * n0000000000000000000, ... n9223372036854775807,
+ * INT64_MIN -1
+ *
+ * o0000000000000000000,
+ * ZERO
+ *
+ * p0000000000000000001, ... p9223372036854775807
+ * +1 INT64_MAX
+ */
+static int ldb_index_format_Integer(struct ldb_context *ldb,
+ void *mem_ctx,
+ const struct ldb_val *in,
+ struct ldb_val *out)
+{
+ int64_t i;
+ int ret;
+ char prefix;
+ size_t len;
+
+ ret = val_to_int64(in, &i);
+ if (ret != LDB_SUCCESS) {
+ return ret;
+ }
+
+ if (i < 0) {
+ /*
+ * i is negative, so this is subtraction rather than
+ * wrap-around.
+ */
+ prefix = 'n';
+ i = INT64_MAX + i + 1;
+ } else if (i > 0) {
+ prefix = 'p';
+ } else {
+ prefix = 'o';
+ }
+
+ out->data = (uint8_t *) talloc_asprintf(mem_ctx, "%c%019lld", prefix, (long long)i);
+ if (out->data == NULL) {
+ ldb_oom(ldb);
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
+
+ len = talloc_array_length(out->data) - 1;
+ if (len != 20) {
+ ldb_debug(ldb, LDB_DEBUG_ERROR,
+ __location__ ": expected index format str %s to"
+ " have length 20 but got %zu",
+ (char*)out->data, len);
+ return LDB_ERR_OPERATIONS_ERROR;
+ }
+
+ out->length = 20;
+ return 0;
+}
+
/*
compare two Integers
*/
@@ -428,7 +535,15 @@ static const struct ldb_schema_syntax ldb_standard_syntaxes[] = {
.canonicalise_fn = ldb_canonicalise_Integer,
.comparison_fn = ldb_comparison_Integer
},
- {
+ {
+ .name = LDB_SYNTAX_ORDERED_INTEGER,
+ .ldif_read_fn = ldb_handler_copy,
+ .ldif_write_fn = ldb_handler_copy,
+ .canonicalise_fn = ldb_canonicalise_Integer,
+ .index_format_fn = ldb_index_format_Integer,
+ .comparison_fn = ldb_comparison_Integer
+ },
+ {
.name = LDB_SYNTAX_OCTET_STRING,
.ldif_read_fn = ldb_handler_copy,
.ldif_write_fn = ldb_handler_copy,
diff --git a/lib/ldb/include/ldb.h b/lib/ldb/include/ldb.h
index 81bee934da5..7ee6a44dd5e 100644
--- a/lib/ldb/include/ldb.h
+++ b/lib/ldb/include/ldb.h
--
Samba Shared Repository
More information about the samba-cvs
mailing list