[SCM] Samba Shared Repository - branch master updated
Andrew Bartlett
abartlet at samba.org
Wed May 23 03:12:03 UTC 2018
The branch, master has been updated
via 754a840 autobuild: build ldb --without-ldb-lmdb
via 1118fc3 selftest: Run ad_dc and vampire_dc with --backend-store=mdb
via 39b7f1b ldb-samba: Handle generic mdb:// url scheme in ldb_relative_path()
via cb5da7a ldb: Reject a possible future ldb_mdb with the index in a sub-database
via 866af32 ldb: Add MDB support to ldb://
via aeeab17 ldb_mdb/tests: add tests for multiple opens across forks
via 04884a8 ldb_mdb/tests: test large index key value
via 65f6ce7 ldb_mdb: Remove implicit read lock and remove transaction counter
via d8919d2 ldb_mdb: Run the ldb_mdb_mod_op_test
via be335f1 ldb_mdb/tests: Tests for wrap open
via 4dc4465 ldb_mdb: Use mdb_env_get_fd() to get the FD for fstat() and FD_CLOEXEC
via 322e428 ldb_mdb: prevent MDB_env reuse across forks
via 14f5c75 ldb_mdb: handle EBADE from mdb_env_open
via eb1bc2e ldb_mdb: Wrap mdb_env_open
via f9a12b6 ldb_mdb: Apply LMDB key length restrictions at key-value layer
via 53d9d49 ldb_mdb/tests: Run api and index test also on lmdb
via a5a000b ldb_mdb/tests: Add tests to check for max key length and DB size
via 0d2d1e5 ldb_mdb: Don't allow modify operations on a read only db
via 95d1e47 ldb_mdb: Store pid to change destructor on fork
via e4e6d79 ldb_mdb: Enable LDB_FLG_NOSYNC in ldb_mdb
via 5ec4910 ldb_mdb: Implement the lmdb backend for ldb
from c8d7e4c selftest: Clean up ldb on tearDown from each packet in TrafficEmulatorPacketTests
https://git.samba.org/?p=samba.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 754a840946cae308fb97a2927ea263e29de9e7b4
Author: Andrew Bartlett <abartlet at samba.org>
Date: Mon May 14 14:34:25 2018 +1200
autobuild: build ldb --without-ldb-lmdb
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
Autobuild-User(master): Andrew Bartlett <abartlet at samba.org>
Autobuild-Date(master): Wed May 23 05:11:13 CEST 2018 on sn-devel-144
commit 1118fc3b2608469e6086802be9e6b2065462770f
Author: Andrew Bartlett <abartlet at samba.org>
Date: Mon May 14 12:14:06 2018 +1200
selftest: Run ad_dc and vampire_dc with --backend-store=mdb
This ensures the LMDB backend is tested in make test
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 39b7f1bcf0d3eaf6308b3ebf88e57f52c0d4a68a
Author: Gary Lockyer <gary at catalyst.net.nz>
Date: Tue Mar 6 13:40:21 2018 +1300
ldb-samba: Handle generic mdb:// url scheme in ldb_relative_path()
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit cb5da7a5c610875297a90be7f0b8bcde620990f1
Author: Andrew Bartlett <abartlet at samba.org>
Date: Sat May 19 07:10:15 2018 +1200
ldb: Reject a possible future ldb_mdb with the index in a sub-database
This ensures we do not corrupt such an index by making changes to the
main database without knowing that the index values are now in a
sub-database.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 866af3270db8076898fb69bddd6f3699cac9e263
Author: Gary Lockyer <gary at catalyst.net.nz>
Date: Tue Mar 6 15:11:23 2018 +1300
ldb: Add MDB support to ldb://
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
commit aeeab1753e6d57cbfb352a3d698cda213ac6c6c9
Author: Gary Lockyer <gary at catalyst.net.nz>
Date: Tue Mar 13 15:08:10 2018 +1300
ldb_mdb/tests: add tests for multiple opens across forks
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 04884a80122fbaf83e60da57ec601585af198d99
Author: Gary Lockyer <gary at catalyst.net.nz>
Date: Tue Mar 13 08:14:09 2018 +1300
ldb_mdb/tests: test large index key value
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 65f6ce7a6e187ce45098b7fd88fdcda90b242837
Author: Gary Lockyer <gary at catalyst.net.nz>
Date: Fri Mar 23 11:29:25 2018 +1300
ldb_mdb: Remove implicit read lock and remove transaction counter
The way to know if we are in a transaction is if there is a non-NULL
transaction handle.
This allows the ldb_mdb_kv_ops_test test to be run.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit d8919d2a59e64ebd8f4bfa7c78abfdf865662186
Author: Andrew Bartlett <abartlet at samba.org>
Date: Wed May 9 11:40:36 2018 +1200
ldb_mdb: Run the ldb_mdb_mod_op_test
ldb_mdb is now able to pass the full ldb_mod_op_test when compiled against lmdb.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit be335f1fbc8ec1c4fb7b329138e742e5f2403b77
Author: Gary Lockyer <gary at catalyst.net.nz>
Date: Thu Mar 8 16:47:59 2018 +1300
ldb_mdb/tests: Tests for wrap open
Tests to ensure that the mdb_env wrapping code correctly handles
multiple ldb's point to the same physical database file.
The test_ldb_close_with_multiple_connections tests are in
ldb_mod_op_test due to the utility code it uses from
elsewhere in that test.
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 4dc44659dd290035a5500a13197e9fcee7d9a21b
Author: Andrew Bartlett <abartlet at samba.org>
Date: Thu May 17 13:10:25 2018 +1200
ldb_mdb: Use mdb_env_get_fd() to get the FD for fstat() and FD_CLOEXEC
This ensures we leave the FD behind if we exec() in a child process.
This deliberatly the same as TDB, as we want the same behaviour as
we have come to expect with that backend.
Signed-off-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 322e42818b144249084dc3a8d4f7988cd8fcab3e
Author: Gary Lockyer <gary at catalyst.net.nz>
Date: Tue Mar 13 15:08:10 2018 +1300
ldb_mdb: prevent MDB_env reuse across forks
MDB_env's may not be reused accross forks. Check the pid that the lmdb
structure was created by, and return an error if it is being used by a
different process.
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 14f5c7522caa0006837658a156f090d8dd65bb0d
Author: Gary Lockyer <gary at catalyst.net.nz>
Date: Wed Mar 21 11:38:22 2018 +1300
ldb_mdb: handle EBADE from mdb_env_open
Under some circumstances mdb_env_open returns EBADE, we treat this as
indicating the file is not a valid lmdb format file.
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit eb1bc2ec0922fe6b86140b1a1ee1f368d222646b
Author: Gary Lockyer <gary at catalyst.net.nz>
Date: Wed Mar 7 12:05:34 2018 +1300
ldb_mdb: Wrap mdb_env_open
Wrap mdb_env_open to ensure that we only have one MDB_env opened per
database in each process
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit f9a12b6433f38cfab97c0057d0e9b0d5719d864a
Author: Gary Lockyer <gary at catalyst.net.nz>
Date: Tue Mar 6 15:27:51 2018 +1300
ldb_mdb: Apply LMDB key length restrictions at key-value layer
We need to enforce the GUID index mode so end-users do not get a
supprise in mid-operation and we enforce a max key length of 511 so
that the index key trunctation is done correctly.
Otherwise the DB will appear to work until a very long key (DN or
index) is used, after which it will be sad.
Because the previous ldb_lmdb_test confirmed the key length by
creating a large DN, those tests are re-worked to use the GUID index
mode. In turn, new tests are written that create a special DN around
the maximum key length.
Finally a test is included that demonstrates that adding entries to
the LMDB DB without GUID index mode fails.
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 53d9d4974dda96f2dcbaa29c78ef1407abfe1341
Author: Gary Lockyer <gary at catalyst.net.nz>
Date: Tue Mar 6 09:13:31 2018 +1300
ldb_mdb/tests: Run api and index test also on lmdb
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit a5a000b68ec8875bcc86217a6c32a97fc3d2723f
Author: Gary Lockyer <gary at catalyst.net.nz>
Date: Fri Feb 2 15:30:53 2018 +1300
ldb_mdb/tests: Add tests to check for max key length and DB size
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 0d2d1e5bf07bb6c4f2ac741e0a0287a4eab523a7
Author: Gary Lockyer <gary at catalyst.net.nz>
Date: Wed Mar 7 12:05:34 2018 +1300
ldb_mdb: Don't allow modify operations on a read only db
Signed-off-by: Gary Lockyer <gary at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
Reviewed-by: Stefan Metzmacher <metze at samba.org>
Reviewed-by: Garming Sam <garming at catalyst.net.nz>
commit 95d1e474cfe24ea65f3cd0bca82d531e6f56e363
Author: Garming Sam <garming at catalyst.net.nz>
Date: Mon Mar 5 16:04:03 2018 +1300
ldb_mdb: Store pid to change destructor on fork
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit e4e6d794ee667b84db9bb1f248bf5ed411e5f5be
Author: Garming Sam <garming at catalyst.net.nz>
Date: Thu Mar 1 16:53:07 2018 +1300
ldb_mdb: Enable LDB_FLG_NOSYNC in ldb_mdb
This is used in selftest with 'ldb:nosync = true'.
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
commit 5ec491040c7fb7bf779a421fac900ea47a40d489
Author: Garming Sam <garming at catalyst.net.nz>
Date: Wed Jan 11 17:10:19 2017 +1300
ldb_mdb: Implement the lmdb backend for ldb
Signed-off-by: Garming Sam <garming at catalyst.net.nz>
Reviewed-by: Andrew Bartlett <abartlet at samba.org>
-----------------------------------------------------------------------
Summary of changes:
lib/ldb-samba/ldb_wrap.c | 2 +
lib/ldb/ldb_ldb/ldb_ldb.c | 19 +
lib/ldb/ldb_mdb/ldb_mdb.c | 887 +++++++++++++++++++++
.../{include/ldb_handlers.h => ldb_mdb/ldb_mdb.h} | 67 +-
.../test/main.c => ldb/ldb_mdb/ldb_mdb_init.c} | 20 +-
lib/ldb/ldb_tdb/ldb_cache.c | 16 +-
lib/ldb/ldb_tdb/ldb_tdb.h | 10 +
lib/ldb/tests/ldb_lmdb_size_test.c | 210 +++++
lib/ldb/tests/ldb_lmdb_test.c | 588 ++++++++++++++
lib/ldb/tests/ldb_mod_op_test.c | 173 ++++
lib/ldb/tests/python/api.py | 97 ++-
lib/ldb/tests/python/index.py | 43 +
lib/ldb/tools/ldbdump.c | 126 ++-
lib/ldb/wscript | 156 +++-
script/autobuild.py | 3 +
selftest/target/Samba4.pm | 5 +-
16 files changed, 2370 insertions(+), 52 deletions(-)
create mode 100644 lib/ldb/ldb_mdb/ldb_mdb.c
copy lib/ldb/{include/ldb_handlers.h => ldb_mdb/ldb_mdb.h} (52%)
copy lib/{replace/test/main.c => ldb/ldb_mdb/ldb_mdb_init.c} (69%)
create mode 100644 lib/ldb/tests/ldb_lmdb_size_test.c
create mode 100644 lib/ldb/tests/ldb_lmdb_test.c
Changeset truncated at 500 lines:
diff --git a/lib/ldb-samba/ldb_wrap.c b/lib/ldb-samba/ldb_wrap.c
index da383d2..34148a1 100644
--- a/lib/ldb-samba/ldb_wrap.c
+++ b/lib/ldb-samba/ldb_wrap.c
@@ -356,6 +356,8 @@ int samba_ldb_connect(struct ldb_context *ldb, struct loadparm_context *lp_ctx,
}
if (strncmp("tdb://", base_url, 6) == 0) {
base_url = base_url+6;
+ } else if (strncmp("mdb://", base_url, 6) == 0) {
+ base_url = base_url+6;
} else if (strncmp("ldb://", base_url, 6) == 0) {
base_url = base_url+6;
}
diff --git a/lib/ldb/ldb_ldb/ldb_ldb.c b/lib/ldb/ldb_ldb/ldb_ldb.c
index 3e4398f..a5a3612 100644
--- a/lib/ldb/ldb_ldb/ldb_ldb.c
+++ b/lib/ldb/ldb_ldb/ldb_ldb.c
@@ -19,6 +19,9 @@
*/
#include "ldb_private.h"
#include "../ldb_tdb/ldb_tdb.h"
+#ifdef HAVE_LMDB
+#include "../ldb_mdb/ldb_mdb.h"
+#endif /* HAVE_LMDB */
/*
connect to the database
@@ -50,6 +53,22 @@ static int lldb_connect(struct ldb_context *ldb,
* Don't create the database if it's not there
*/
flags |= LDB_FLG_DONT_CREATE_DB;
+#ifdef HAVE_LMDB
+ /*
+ * Try opening the database as an lmdb
+ */
+ ret = lmdb_connect(ldb, path, flags, options, module);
+ if (ret == LDB_SUCCESS) {
+ return ret;
+ }
+ if (ret != LDB_ERR_UNAVAILABLE) {
+ return ret;
+ }
+
+ /*
+ * Not mdb so try as tdb
+ */
+#endif /* HAVE_LMDB */
ret = ltdb_connect(ldb, path, flags, options, module);
return ret;
}
diff --git a/lib/ldb/ldb_mdb/ldb_mdb.c b/lib/ldb/ldb_mdb/ldb_mdb.c
new file mode 100644
index 0000000..af552fe
--- /dev/null
+++ b/lib/ldb/ldb_mdb/ldb_mdb.c
@@ -0,0 +1,887 @@
+/*
+ ldb database library using mdb back end
+
+ Copyright (C) Jakub Hrozek 2014
+ Copyright (C) Catalyst.Net Ltd 2017
+
+ ** NOTE! The following LGPL license applies to the ldb
+ ** library. This does NOT imply that all of Samba is released
+ ** under the LGPL
+
+ This library is free software; you can redistribute it and/or
+ modify it under the terms of the GNU Lesser General Public
+ License as published by the Free Software Foundation; either
+ version 3 of the License, or (at your option) any later version.
+
+ This library is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ Lesser General Public License for more details.
+
+ You should have received a copy of the GNU Lesser General Public
+ License along with this library; if not, see <http://www.gnu.org/licenses/>.
+*/
+
+#include "ldb_mdb.h"
+#include "../ldb_tdb/ldb_tdb.h"
+#include "include/dlinklist.h"
+
+#define MDB_URL_PREFIX "mdb://"
+#define MDB_URL_PREFIX_SIZE (sizeof(MDB_URL_PREFIX)-1)
+
+#define LDB_MDB_MAX_KEY_LENGTH 511
+
+#define GIGABYTE (1024*1024*1024)
+
+int ldb_mdb_err_map(int lmdb_err)
+{
+ switch (lmdb_err) {
+ case MDB_SUCCESS:
+ return LDB_SUCCESS;
+ case EIO:
+ return LDB_ERR_OPERATIONS_ERROR;
+ case EBADE:
+ case MDB_INCOMPATIBLE:
+ case MDB_CORRUPTED:
+ case MDB_INVALID:
+ return LDB_ERR_UNAVAILABLE;
+ case MDB_BAD_TXN:
+ case MDB_BAD_VALSIZE:
+#ifdef MDB_BAD_DBI
+ case MDB_BAD_DBI:
+#endif
+ case MDB_PANIC:
+ case EINVAL:
+ return LDB_ERR_PROTOCOL_ERROR;
+ case MDB_MAP_FULL:
+ case MDB_DBS_FULL:
+ case MDB_READERS_FULL:
+ case MDB_TLS_FULL:
+ case MDB_TXN_FULL:
+ case EAGAIN:
+ return LDB_ERR_BUSY;
+ case MDB_KEYEXIST:
+ return LDB_ERR_ENTRY_ALREADY_EXISTS;
+ case MDB_NOTFOUND:
+ case ENOENT:
+ return LDB_ERR_NO_SUCH_OBJECT;
+ case EACCES:
+ return LDB_ERR_INSUFFICIENT_ACCESS_RIGHTS;
+ default:
+ break;
+ }
+ return LDB_ERR_OTHER;
+}
+
+#define ldb_mdb_error(ldb, ecode) lmdb_error_at(ldb, ecode, __FILE__, __LINE__)
+static int lmdb_error_at(struct ldb_context *ldb,
+ int ecode,
+ const char *file,
+ int line)
+{
+ int ldb_err = ldb_mdb_err_map(ecode);
+ char *reason = mdb_strerror(ecode);
+ ldb_asprintf_errstring(ldb,
+ "(%d) - %s at %s:%d",
+ ecode,
+ reason,
+ file,
+ line);
+ return ldb_err;
+}
+
+
+static bool lmdb_transaction_active(struct ltdb_private *ltdb)
+{
+ return ltdb->lmdb_private->txlist != NULL;
+}
+
+static MDB_txn *lmdb_trans_get_tx(struct lmdb_trans *ltx)
+{
+ if (ltx == NULL) {
+ return NULL;
+ }
+
+ return ltx->tx;
+}
+
+static void trans_push(struct lmdb_private *lmdb, struct lmdb_trans *ltx)
+{
+ if (lmdb->txlist) {
+ talloc_steal(lmdb->txlist, ltx);
+ }
+
+ DLIST_ADD(lmdb->txlist, ltx);
+}
+
+static void trans_finished(struct lmdb_private *lmdb, struct lmdb_trans *ltx)
+{
+ DLIST_REMOVE(lmdb->txlist, ltx);
+ talloc_free(ltx);
+}
+
+
+static struct lmdb_trans *lmdb_private_trans_head(struct lmdb_private *lmdb)
+{
+ struct lmdb_trans *ltx;
+
+ ltx = lmdb->txlist;
+ return ltx;
+}
+
+
+static MDB_txn *get_current_txn(struct lmdb_private *lmdb)
+{
+ MDB_txn *txn = NULL;
+
+ txn = lmdb_trans_get_tx(lmdb_private_trans_head(lmdb));
+ if (txn != NULL) {
+ return txn;
+ }
+ if (lmdb->read_txn != NULL) {
+ return lmdb->read_txn;
+ }
+ lmdb->error = MDB_BAD_TXN;
+ ldb_set_errstring(lmdb->ldb, __location__":No active transaction\n");
+ return NULL;
+}
+
+static int lmdb_store(struct ltdb_private *ltdb,
+ struct ldb_val key,
+ struct ldb_val data, int flags)
+{
+ struct lmdb_private *lmdb = ltdb->lmdb_private;
+ MDB_val mdb_key;
+ MDB_val mdb_data;
+ int mdb_flags;
+ MDB_txn *txn = NULL;
+ MDB_dbi dbi = 0;
+
+ if (ltdb->read_only) {
+ return LDB_ERR_UNWILLING_TO_PERFORM;
+ }
+
+ txn = lmdb_trans_get_tx(lmdb_private_trans_head(lmdb));
+ if (txn == NULL) {
+ ldb_debug(lmdb->ldb, LDB_DEBUG_FATAL, "No transaction");
+ lmdb->error = MDB_PANIC;
+ return ldb_mdb_error(lmdb->ldb, lmdb->error);
+ }
+
+ lmdb->error = mdb_dbi_open(txn, NULL, 0, &dbi);
+ if (lmdb->error != MDB_SUCCESS) {
+ return ldb_mdb_error(lmdb->ldb, lmdb->error);
+ }
+
+ mdb_key.mv_size = key.length;
+ mdb_key.mv_data = key.data;
+
+ mdb_data.mv_size = data.length;
+ mdb_data.mv_data = data.data;
+
+ if (flags == TDB_INSERT) {
+ mdb_flags = MDB_NOOVERWRITE;
+ } else if ((flags == TDB_MODIFY)) {
+ /*
+ * Modifying a record, ensure that it exists.
+ * This mimics the TDB semantics
+ */
+ MDB_val value;
+ lmdb->error = mdb_get(txn, dbi, &mdb_key, &value);
+ if (lmdb->error != MDB_SUCCESS) {
+ return ldb_mdb_error(lmdb->ldb, lmdb->error);
+ }
+ mdb_flags = 0;
+ } else {
+ mdb_flags = 0;
+ }
+
+ lmdb->error = mdb_put(txn, dbi, &mdb_key, &mdb_data, mdb_flags);
+ if (lmdb->error != MDB_SUCCESS) {
+ return ldb_mdb_error(lmdb->ldb, lmdb->error);
+ }
+
+ return ldb_mdb_err_map(lmdb->error);
+}
+
+static int lmdb_delete(struct ltdb_private *ltdb, struct ldb_val key)
+{
+ struct lmdb_private *lmdb = ltdb->lmdb_private;
+ MDB_val mdb_key;
+ MDB_txn *txn = NULL;
+ MDB_dbi dbi = 0;
+
+ if (ltdb->read_only) {
+ return LDB_ERR_UNWILLING_TO_PERFORM;
+ }
+
+ txn = lmdb_trans_get_tx(lmdb_private_trans_head(lmdb));
+ if (txn == NULL) {
+ ldb_debug(lmdb->ldb, LDB_DEBUG_FATAL, "No transaction");
+ lmdb->error = MDB_PANIC;
+ return ldb_mdb_error(lmdb->ldb, lmdb->error);
+ }
+
+ lmdb->error = mdb_dbi_open(txn, NULL, 0, &dbi);
+ if (lmdb->error != MDB_SUCCESS) {
+ return ldb_mdb_error(lmdb->ldb, lmdb->error);
+ }
+
+ mdb_key.mv_size = key.length;
+ mdb_key.mv_data = key.data;
+
+ lmdb->error = mdb_del(txn, dbi, &mdb_key, NULL);
+ if (lmdb->error != MDB_SUCCESS) {
+ return ldb_mdb_error(lmdb->ldb, lmdb->error);
+ }
+ return ldb_mdb_err_map(lmdb->error);
+}
+
+static int lmdb_traverse_fn(struct ltdb_private *ltdb,
+ ldb_kv_traverse_fn fn,
+ void *ctx)
+{
+ struct lmdb_private *lmdb = ltdb->lmdb_private;
+ MDB_val mdb_key;
+ MDB_val mdb_data;
+ MDB_txn *txn = NULL;
+ MDB_dbi dbi = 0;
+ MDB_cursor *cursor = NULL;
+ int ret;
+
+ txn = get_current_txn(lmdb);
+ if (txn == NULL) {
+ ldb_debug(lmdb->ldb, LDB_DEBUG_FATAL, "No transaction");
+ lmdb->error = MDB_PANIC;
+ return ldb_mdb_error(lmdb->ldb, lmdb->error);
+ }
+
+ lmdb->error = mdb_dbi_open(txn, NULL, 0, &dbi);
+ if (lmdb->error != MDB_SUCCESS) {
+ return ldb_mdb_error(lmdb->ldb, lmdb->error);
+ }
+
+ lmdb->error = mdb_cursor_open(txn, dbi, &cursor);
+ if (lmdb->error != MDB_SUCCESS) {
+ goto done;
+ }
+
+ while ((lmdb->error = mdb_cursor_get(
+ cursor, &mdb_key,
+ &mdb_data, MDB_NEXT)) == MDB_SUCCESS) {
+
+ struct ldb_val key = {
+ .length = mdb_key.mv_size,
+ .data = mdb_key.mv_data,
+ };
+ struct ldb_val data = {
+ .length = mdb_data.mv_size,
+ .data = mdb_data.mv_data,
+ };
+
+ ret = fn(ltdb, key, data, ctx);
+ if (ret != 0) {
+ goto done;
+ }
+ }
+ if (lmdb->error == MDB_NOTFOUND) {
+ lmdb->error = MDB_SUCCESS;
+ }
+done:
+ if (cursor != NULL) {
+ mdb_cursor_close(cursor);
+ }
+
+ if (lmdb->error != MDB_SUCCESS) {
+ return ldb_mdb_error(lmdb->ldb, lmdb->error);
+ }
+ return ldb_mdb_err_map(lmdb->error);
+}
+
+static int lmdb_update_in_iterate(struct ltdb_private *ltdb,
+ struct ldb_val key,
+ struct ldb_val key2,
+ struct ldb_val data,
+ void *state)
+{
+ struct lmdb_private *lmdb = ltdb->lmdb_private;
+ struct ldb_val copy;
+ int ret = LDB_SUCCESS;
+
+ /*
+ * Need to take a copy of the data as the delete operation alters the
+ * data, as it is in private lmdb memory.
+ */
+ copy.length = data.length;
+ copy.data = talloc_memdup(ltdb, data.data, data.length);
+ if (copy.data == NULL) {
+ lmdb->error = MDB_PANIC;
+ return ldb_oom(lmdb->ldb);
+ }
+
+ lmdb->error = lmdb_delete(ltdb, key);
+ if (lmdb->error != MDB_SUCCESS) {
+ ldb_debug(
+ lmdb->ldb,
+ LDB_DEBUG_ERROR,
+ "Failed to delete %*.*s "
+ "for rekey as %*.*s: %s",
+ (int)key.length, (int)key.length,
+ (const char *)key.data,
+ (int)key2.length, (int)key2.length,
+ (const char *)key.data,
+ mdb_strerror(lmdb->error));
+ ret = ldb_mdb_error(lmdb->ldb, lmdb->error);
+ goto done;
+ }
+
+ lmdb->error = lmdb_store(ltdb, key2, copy, 0);
+ if (lmdb->error != MDB_SUCCESS) {
+ ldb_debug(
+ lmdb->ldb,
+ LDB_DEBUG_ERROR,
+ "Failed to rekey %*.*s as %*.*s: %s",
+ (int)key.length, (int)key.length,
+ (const char *)key.data,
+ (int)key2.length, (int)key2.length,
+ (const char *)key.data,
+ mdb_strerror(lmdb->error));
+ ret = ldb_mdb_error(lmdb->ldb, lmdb->error);
+ goto done;
+ }
+
+done:
+ if (copy.data != NULL) {
+ TALLOC_FREE(copy.data);
+ copy.length = 0;
+ }
+
+ /*
+ * Explicity invalidate the data, as the delete has done this
+ */
+ data.length = 0;
+ data.data = NULL;
+
+ return ret;
+}
+
+/* Handles only a single record */
+static int lmdb_parse_record(struct ltdb_private *ltdb, struct ldb_val key,
+ int (*parser)(struct ldb_val key, struct ldb_val data,
+ void *private_data),
+ void *ctx)
+{
+ struct lmdb_private *lmdb = ltdb->lmdb_private;
+ MDB_val mdb_key;
+ MDB_val mdb_data;
+ MDB_txn *txn = NULL;
+ MDB_dbi dbi;
+ struct ldb_val data;
+
+ txn = get_current_txn(lmdb);
+ if (txn == NULL) {
+ ldb_debug(lmdb->ldb, LDB_DEBUG_FATAL, "No transaction active");
+ lmdb->error = MDB_PANIC;
+ return ldb_mdb_error(lmdb->ldb, lmdb->error);
+ }
+
+ lmdb->error = mdb_dbi_open(txn, NULL, 0, &dbi);
+ if (lmdb->error != MDB_SUCCESS) {
+ return ldb_mdb_error(lmdb->ldb, lmdb->error);
+ }
+
+ mdb_key.mv_size = key.length;
+ mdb_key.mv_data = key.data;
+
+ lmdb->error = mdb_get(txn, dbi, &mdb_key, &mdb_data);
+ if (lmdb->error != MDB_SUCCESS) {
+ /* TODO closing a handle should not even be necessary */
+ mdb_dbi_close(lmdb->env, dbi);
+ if (lmdb->error == MDB_NOTFOUND) {
+ return LDB_ERR_NO_SUCH_OBJECT;
+ }
+ return ldb_mdb_error(lmdb->ldb, lmdb->error);
+ }
+ data.data = mdb_data.mv_data;
+ data.length = mdb_data.mv_size;
+
+ /* TODO closing a handle should not even be necessary */
+ mdb_dbi_close(lmdb->env, dbi);
+
+ return parser(key, data, ctx);
+}
+
+
+static int lmdb_lock_read(struct ldb_module *module)
+{
+ void *data = ldb_module_get_private(module);
+ struct ltdb_private *ltdb = talloc_get_type(data, struct ltdb_private);
+ struct lmdb_private *lmdb = ltdb->lmdb_private;
+ pid_t pid = getpid();
+
+ if (pid != lmdb->pid) {
+ ldb_asprintf_errstring(
+ lmdb->ldb,
+ __location__": Reusing ldb opened by pid %d in "
+ "process %d\n",
+ lmdb->pid,
+ pid);
+ lmdb->error = MDB_BAD_TXN;
+ return LDB_ERR_PROTOCOL_ERROR;
+ }
+
+ lmdb->error = MDB_SUCCESS;
+ if (lmdb_transaction_active(ltdb) == false &&
+ ltdb->read_lock_count == 0) {
+ lmdb->error = mdb_txn_begin(lmdb->env,
+ NULL,
+ MDB_RDONLY,
+ &lmdb->read_txn);
+ }
+ if (lmdb->error != MDB_SUCCESS) {
+ return ldb_mdb_error(lmdb->ldb, lmdb->error);
+ }
+
--
Samba Shared Repository
More information about the samba-cvs
mailing list