dbwrap_py: rapid dbwrap backend prototyping and FoundationDB

Jeremy Allison jra at samba.org
Tue Jan 17 20:17:06 UTC 2023

On Tue, Jan 17, 2023 at 03:33:42PM +0100, Ralph Boehme wrote:
>I've hacked a new dbwrap backend called dbwrap_py that calls into 
>Python for the database operations. This came out of the desire to 
>play around with the many available distributed database that are 
>available these days, eg Cassandra, etcd, FoundationDB and so on [1]. 
>All of them have Python bindings and implementing the required 
>primitives in Python is much simpler then doing it in C.
>I've developed dbwrap_py hand in hand with an initial backend 
>targetting FoundationDB which is a scalable, high-performance 
>key-value store with strong consistency and transaction support
>Thanks to the transaction support in FoundationDB it is possible to 
>implement locking using a dedicated locking record per record combined 
>with a fencing token that guards modfying database operations. Other 
>databases may offer other primitives to implement record locking.
>The whole thing is now functional enough such that it passes certain 
>torture tests when configured to put locking.tdb and 
>smbXsrv_open_global.tdb behind dbwrap_py.
>Currently the whole dbwrap layering runs unchanged on top of the 
>backend. As locking is implemented in the backend and FoundationDB 
>support watches, it would be possible to not use the current g_lock 
>and dbwrap_watch backend for dbs enabled for FoundationDB.
>With metze's new smb2.bench.path-contention-shared benchmark I'm 
>getting 25 opens/closes per second on my laptop with FoundationDB 
>being a single node cluster running on the laptop as well:
>$ bin/smbtorture //localhost/test -U slow%Passw0rd 
>Opened 4 connections with qdepth=1 => 4 loops
>Running for 10 seconds
>$ bin/smbtorture3 //localhost/test -U slow%Passw0rd LOCAL-DBWRAP-PY
>dbwrap_fetch_locked() took: 2799 us
>dbwrap_record_store() took: 4205 us
>dbwrap_exists() took: 364 us
>dbwrap_parse_record() took: 3571 us
>dbwrap_do_locked() took: 5552 us
>The code is here:
>    dbwrap_py:module = samba.samba3.dbwrap_py_fdb
>    dbwrap_py:locking.tdb = yes
>    dbwrap_py:smbXsrv_open_global.tdb = yes
>    dbwrap_py:test_db_py.tdb = yes
>FoundationDB would be a good candidate for a public cloud SMB frontend 
>ala Azure SMB, this is not aiming at the ctdb high performance SMB NAS 
>I'll try to get a small FoundationDB cluster setup in K8 to see how 
>that feels like. Stay tune... ;)

Great work Ralph ! I'm looking forward to playing with this.


More information about the samba-technical mailing list