LMDB key value backend for ldb_tdb (to be renamed ldb_key_val)
garming at catalyst.net.nz
Mon Mar 5 22:12:01 UTC 2018
On 22/02/18 11:45, Garming Sam wrote:
> Performance numbers at this point seem a bit tricky to obtain. Our
> existing perf testing infrastructure relies on the test environment,
> however, LMDB appears to run noticeably slower under cwrap due to some
> calls being intercepted. Basic testing indicates better baseline figures
> and better concurrency (reads no longer block writes, and writes
> effectively only block reads during commit - not prepare commit), but
> there needs to be a lot more testing to properly understand the
> performance characteristics.
> Somewhat related, I am currently investigating running our tests without
> socket wrapper by using network namespaces. There are still areas where
> being able to run socket wrapper is definitely useful, but performance
> testing is definitely not one of them.
Findings so far:
Socket wrapper has a roughly 20% overhead with pure adds when using LMDB
(compared to using a network namespace) with some NOSYNC options set.
Although I find TDB incurs a similar overhead for running under
namespaces strangely enough, which I don't have an adequate explanation for.
Some further testing seems to indicate that LMDB performance is on
parity with TDB performance with writes, and has improved performance
for searches as the database grows larger (5-10% slower with no added
users, 10-20% faster on an unindexed search with around 5000 users). The
other refactorings required for the key value store also appear to have
little to no performance impact on TDB.
More information about the samba-technical