tdb API issues
hyc at highlandsun.com
Thu Apr 2 00:20:55 GMT 2009
tridge at samba.org wrote:
> Hi Howard,
> The error code isn't the only place where tdb doesn't mix well with
> threads. Things like tdb_traverse() involve state in the tdb context,
Ah, I wasn't planning to use tdb_traverse() so I hadn't even looked at that.
It would probably require splitting the traverse state out of the tdb_context
and into its own dynamically allocated cursor struct.
> and all the tdb_lock primitives would affect the whole process, not
> just a thread.
I didn't expect this would be a problem since normally only one slapd process
accesses a DB at a time. (With the exception that slapcat is allowed read-only
access.) I figured using a rdwr lock to control calls into tdb should be
sufficient. Did I miss something?
> The same goes for the transaction code. For a threaded environment,
> you'd like it if it worked so that when thread1 was in a transaction,
> that thread2 would be blocked in writes, and reads would see the
> pre-transaction data (this is what happens with two processes in tdb).
Yes, I was thinking that we'd be getting dirty reads when I looked at this
code; I'd forgotten that it was protecting against other processes and not
other threads. So we'd need a way to make sure only the thread that invoked
the transaction used transaction_read(), while all the other threads still
fell into the default tdb_read(). Sounds like we need an actual transaction
handle as well, instead of just leaving all this in the tdb_context.
> If you can arrange it so that each thread has its own tdb context then
> life will be much simpler. That gives you a separate ecode per thread
> as well. The problem is that you'll probably need to use some other
> locking mechanism than fcntl locks, so you'll need to offer a hook
> that abstracts away the locking primitives. Trying to make this work
> so that tdb operations can also be performed safely by other processes
> on the same tdb that is being used by your threaded task will be
> interesting (I think its possible, just a bit tricky).
I'd thought of that at first, but it seems that a separate context per thread
would also multiply the address space consumed per DB, since they would each
have their own complete mmap of the data. Doesn't sound too practical.
Yeah, coordinating the fcntl locks across multiple tdb_context's would also be
a pain. Pretty sure that splitting to a separate context per thread is the
wrong level of granularity... ;)
> tdb and threads certainly isn't easy :-)
> Cheers, Tridge
-- Howard Chu
CTO, Symas Corp. http://www.symas.com
Director, Highland Sun http://highlandsun.com/hyc/
Chief Architect, OpenLDAP http://www.openldap.org/project/
More information about the samba-technical