TDB2: Draft Design Document

[Rusty Russell]

Hi all,

    Sorry for the delay.  Volker suggested a while back that I jot down my
thoughts on enhancing TDB, and the results are attached.

Quick overview:

1 Introduction
2 API Issues
 2.1 tdb_open_ex Is Not Expandable
 2.2 tdb_traverse Makes Impossible Guarantees
 2.3 Nesting of Transactions Is Fraught
 2.4 Incorrect Hash Function is Not Detected
 2.5 tdb_set_max_dead/TDB_VOLATILE Expose Implementation
 2.6 TDB Files Cannot Be Opened Multiple Times 
 2.7 TDB API Is Not POSIX Thread-safe
 2.8 *_nonblock Functions And *_mark Functions Expose 
 2.9 tdb_chainlock Functions Expose Implementation
 2.10 Signal Handling is Not Race-Free
 2.11 The API Uses Gratuitous Typedefs, Capitals
 2.12 tdb_log_func Doesn't Take The 
 2.13 Various Callback Functions Are Not Typesafe
 2.14 TDB_CLEAR_IF_FIRST Must Be Specified On All Opens, 
3 Performance and Scalability
 3.1 TDB_CLEAR_IF_FIRST 
 3.2 TDB Files Have a 4G Limit
 3.3 TDB Records Have a 4G Limit
 3.4 Hash Size Is Determined At TDB Creation Time
 3.5 TDB Freelist Is Highly Contended
 3.6 TDB Becomes Fragmented
 3.7 Records Incur A 28-Byte Overhead
 3.8 Transaction Commit Requires 4 fdatasync
 3.9 TDB Does Not Have Snapshot Support
 3.10 Transactions Cannot Operate in Parallel
 3.11 Default Hash Function Is Suboptimal
 3.12 Reliable Traversal Adds Complexity
 3.13 Fcntl Locking Adds Overhead
 3.14 Some Transactions Don't Require Durability

Cheers,
Rusty.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: design.txt.bz2
Type: application/x-bzip
Size: 13031 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20100802/32fce9cf/attachment-0003.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: design.pdf.bz2
Type: application/x-bzpdf
Size: 117009 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20100802/32fce9cf/attachment-0004.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: design.lyx.bz2
Type: application/x-bzip
Size: 13632 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20100802/32fce9cf/attachment-0005.bin>