LMDB key value backend for ldb_tdb (to be renamed ldb_key_val)

[Garming Sam]

On 17/04/18 01:11, Howard Chu via samba-technical wrote:
> From: William Brown <william at blackhats.net.au>
>>> > re-thinking the overall architecture e.g. consolidating the
>>> > partitions
>>> > into a single file using LMDB sub databases.
>>
>> There are some arguments both ways here I think.
>>
>> Keeping them seperate means that you can easily identify any component
>> that has a size blow out (specific index, changelog etc), and
>> potentially resolve it. This is pretty nice for admins to be able to
>> deal with. It also makes it easier to export/import large files to
>> resize them (lmdb files can't be resized iirc). If you have a combined
>> DB and it's say ... 8GB, then you have to have at least 8GB free to
>> export and import to a new db. If this was spilt to many files (say
>> 0.5GB each) it becomes easier to manage this.
>
> mdb_stat will tell you the sizes of each subDB. You can use "mdb_dump
> -s" to export a specific subDB. And "mdb_drop -s" to delete a specific
> subDB. Not sure what you mean about "LMDB files can't be resized" -
> sure they can, that's what mdb_env_set_mapsize() does.
>

>From Gary attempting an automatic resize mechanism, the caveats
associated with mdb_env_set_mapsize() made it not straightforward to
use, at least within the context of LDB. I presume William ran into
similar complications. Right now, we've only used a fixed figure, but
with the next iteration of our work, we'll probably be looking at how to
get this right.


Cheers,

Garming

>> However, joining these would be good for simpler transaction
>> managament, especially to prevent deadlocking if tables are locked out-
>> of-order. 
>