[SCM] CTDB repository - branch master updated - ctdb-1.0.114-66-g35d9ac7
Ronnie Sahlberg
sahlberg at samba.org
Thu Apr 22 17:25:37 MDT 2010
The branch, master has been updated
via 35d9ac7d162f99795ac3146fcd464e9d2e45feaa (commit)
via 54bee0ec670724c337b882e7dda3e999ffa80691 (commit)
via e174dc084f11db0eb239b643affef2c02c711b1c (commit)
via f1242d9cba676d1d393375bea42613192c70662e (commit)
via e61ff181085b35cb4aba933350bfe4f64471b359 (commit)
via 87ced00d6d98be4a34719af58694e7c940b4dd68 (commit)
via fa38f818c71c85918e673ff563bf7a91a0c4cc17 (commit)
via 7161cb1607bb105cd6f4f32df50f519314e77b3f (commit)
via 7f1330adc690360a4a8ce85352df0908a93684a5 (commit)
via 3318c74386e202b77ba7e29b4c3e0ce4d88fcd5b (commit)
via e90aba9967ea4a8ae7f6bdfc19666c47bd92951e (commit)
via 4215c7025d2b29439c5acd19ce4e0fc4e67370b3 (commit)
via 52b0f19636565ef633e63d2726a1cc8c41dccedb (commit)
via e5af0ce79c74f11daae2a8514e398c40535e2e67 (commit)
via 65bc926d1a9cb3af18cae6b1462b832a5bcec561 (commit)
via 6d7093cf51d0256245cc6bab24c9550ed3f1d8a5 (commit)
via 803035716338170896fee15f15b17c32e7ee777e (commit)
via 74874ffb2c81e098c1d7935b37557c2151382ca6 (commit)
via d7fdb38ac05b5f2af9eb485e98673280835273dd (commit)
via 7a99cdf5d0a91764a750c1a264e90e5b66f910a1 (commit)
via 46f2c33357c999c31a8064c159c6162269c28d9d (commit)
via 587ac01ce836286aab54bfcb7a693a0170c7ebd3 (commit)
via 2e8512403525c14c9b776ce28891d09c17ada91d (commit)
via 0a44584963232b0b1c62e30c9bede0439e68ef7d (commit)
via 6812d81907793299e874f121174d885f6500f374 (commit)
via 4ca1b96a70048b2eaa0d12fb5f0fdb54ec396aa3 (commit)
via cfb154dd0f189f37b937e90144c2eb9e66a26420 (commit)
via d2dd720b51c4032e5d77d30212da8117d3f119df (commit)
via dda3587dfee598f387c2e696f3645486fac65052 (commit)
via 495f3554259b9dbf9ee673c4fe420d98e50e4901 (commit)
via 28576ddbd9bf91049db8a4f9e9e7856ac5b8f48a (commit)
via a912657fb50a78b9b328c4564281fb9f7f1b3766 (commit)
via a4f83910a485cf56f9b3df1dcf2ad36ebec57473 (commit)
via d728a7f65bcd5f1aedbee41d6db5c35f10cb417e (commit)
via 7b5fdc9c588237c83a1e70e5437e2e5510055b92 (commit)
via b0dff4ed35ab2423b8fcc801cdaaebaa0d7654bb (commit)
via e9ee4aaeb471a7f5ba4c97d3f76f406c1fe9b92f (commit)
via 8b0a8a96edf8611257e58ea81ed872dc03ca5da6 (commit)
via 8c8782ecbb347e026f67d82a39555c0e43b1e9f8 (commit)
via e0ca2e02120258aabca1e1586a58a8d672484fb5 (commit)
via 8885b2206fba41ec289fda5dfd653ee676aa0dd3 (commit)
via 84c971f33c24d32e5599aba7ba83bb474f7ac922 (commit)
from 83446b2e7e28e3ed6627c1950053018b8799984a (commit)
http://gitweb.samba.org/?p=sahlberg/ctdb.git;a=shortlog;h=master
- Log -----------------------------------------------------------------
commit 35d9ac7d162f99795ac3146fcd464e9d2e45feaa
Merge: 83446b2e7e28e3ed6627c1950053018b8799984a 54bee0ec670724c337b882e7dda3e999ffa80691
Author: Ronnie sahlberg <ronniesahlberg at gmail.com>
Date: Fri Apr 23 09:25:25 2010 +1000
Merge commit 'rusty/tdb-update'
commit 54bee0ec670724c337b882e7dda3e999ffa80691
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 14:11:38 2010 +0930
tdb: define _PUBLIC_ so we can compile tdb.
The Samba tree defines _PUBLIC_ (and _PRIVATE_) for libraries to
control visibility. The last commit absorbed this from their tdb,
but we need to #define to stub it out since ctdb doesn't use it
(and doesn't need to: we only use tdb internally).
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
commit e174dc084f11db0eb239b643affef2c02c711b1c
Author: Andrew Tridgell <tridge at samba.org>
Date: Thu Apr 22 14:01:36 2010 +0930
tdb: update tdb ABI to use hide_symbols=True
We now use -fvisibilty=hidden to hide symbols from outside the tdb
shared library.
This also moved tdb_transaction_recover() into the tdb_private.h
header, as it should never have been a public API. For that reason we
are changing the version number. We're only doing a minor version
increment as it is extremely unlikely that anyone was actually using
tdb_transaction_recover() as its locking requirements were rather
unusual.
Pair-Programmed-With: Rusty Russell <rusty at samba.org>
(Imported from commit 773a8afbba27a5e2e48577100f3ca9873b506615)
commit f1242d9cba676d1d393375bea42613192c70662e
Author: Jelmer Vernooij <jelmer at samba.org>
Date: Thu Apr 22 13:59:22 2010 +0930
subunit: Support formatting compatible with upstream subunit, for consistency.
Upstream subunit makes a ":" after commands optional, so I've fixed any
places where we might trigger commands accidently. I've filed a bug
about this in subunit.
(Imported from commit 7da94cc4a664521be279b019e9f32121cd410193)
commit e61ff181085b35cb4aba933350bfe4f64471b359
Author: Simo Sorce <idra at samba.org>
Date: Thu Apr 22 13:58:35 2010 +0930
tdb: update exports and signatures files
(Imported from commit c1f6f61f620e865516d1856c9d937b5326a29046)
commit 87ced00d6d98be4a34719af58694e7c940b4dd68
Author: Volker Lendecke <vl at samba.org>
Date: Thu Apr 22 13:58:35 2010 +0930
tdb: Add a non-blocking version of tdb_transaction_start
(Imported from commit 261c3b4f1beed820647061bacbee3acccbcbb089)
commit fa38f818c71c85918e673ff563bf7a91a0c4cc17
Author: Volker Lendecke <vl at samba.org>
Date: Thu Apr 22 13:58:07 2010 +0930
tdb: Fix indentation in tdb_new_database()
(Imported from commit 59315887a07033316edf91c0c57563eee5ea992d)
commit 7161cb1607bb105cd6f4f32df50f519314e77b3f
Author: Volker Lendecke <vl at samba.org>
Date: Thu Apr 22 13:58:07 2010 +0930
Fix some nonempty blank lines
(Imported from commit ea8e0d5d54b020c530e392c4edaeed43e20af303)
commit 7f1330adc690360a4a8ce85352df0908a93684a5
Author: Andrew Tridgell <tridge at samba.org>
Date: Thu Apr 22 13:57:17 2010 +0930
python: use '#!/usr/bin/env python' to cope with varying install locations
this should be much more portable
(Imported from commit 088096d1bad51428a2e2d487214995d4fdfc7ccc)
commit 3318c74386e202b77ba7e29b4c3e0ce4d88fcd5b
Author: Volker Lendecke <vl at samba.org>
Date: Thu Apr 22 13:54:06 2010 +0930
tdb: Fix bug 7248, avoid the nanosleep dependency
(Imported from commit e2c7e5c4f72565fe49265d5b036531926ea1ac92)
commit e90aba9967ea4a8ae7f6bdfc19666c47bd92951e
Author: Volker Lendecke <vl at samba.org>
Date: Thu Apr 22 13:54:06 2010 +0930
tdb: If tdb_parse_record does not find a record, return -1 instead of 0
(Imported from commit fb98f60594b6cabc52d0f2f49eda08f793ba4748)
commit 4215c7025d2b29439c5acd19ce4e0fc4e67370b3
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:54:06 2010 +0930
tdb: handle processes dying during transaction commit.
tdb transactions were designed to be robust against the machine
powering off, but interestingly were never designed to handle the case
where an administrator kill -9's a process during commit. Because
recovery is only done on tdb_open, processes with the tdb already
mapped will simply use it despite it being corrupt and needing
recovery.
The solution to this is to check for recovery every time we grab a
data lock: we could have gained the lock because a process just died.
This has no measurable cost: here is the time for tdbtorture -s 0 -n 1
-l 10000:
Before:
2.75 2.50 2.81 3.19 2.91 2.53 2.72 2.50 2.78 2.77 = Avg 2.75
After:
2.81 2.57 3.42 2.49 3.02 2.49 2.84 2.48 2.80 2.43 = Avg 2.74
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit ec96ea690edbe3398d690b4a953d487ca1773f1c)
commit 52b0f19636565ef633e63d2726a1cc8c41dccedb
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:54:06 2010 +0930
patch tdb-refactor-tdb_lock-and-tdb_lock_nonblock.patch
(Imported from commit 1bf482b9ef9ec73dd7ee4387d7087aa3955503dd)
commit e5af0ce79c74f11daae2a8514e398c40535e2e67
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:54:06 2010 +0930
tdb: add -k option to tdbtorture
To test the case of death of a process during transaction commit, add
a -k (kill random) option to tdbtorture. The easiest way to do this
is to make every worker a child (unless there's only one child), which
is why this patch is bigger than you might expect.
Using -k without -t (always transactions) you expect corruption, though
it doesn't happen every time. With -t, we currently get corruption but
the next patch fixes that.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit ececeffd85db1b27c07cdf91a921fd203006daf6)
commit 65bc926d1a9cb3af18cae6b1462b832a5bcec561
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:54:06 2010 +0930
tdb: don't truncate tdb on recovery
The current recovery code truncates the tdb file on recovery. This is
fine if recovery is only done on first open, but is a really bad idea
as we move to allowing recovery on "live" databases.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit 8c3fda4318adc71899bc41486d5616da3a91a688)
commit 6d7093cf51d0256245cc6bab24c9550ed3f1d8a5
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:54:06 2010 +0930
tdb: remove lock ops
Now the transaction code uses the standard allrecord lock, that stops
us from trying to grab any per-record locks anyway. We don't need to
have special noop lock ops for transactions.
This is a nice simplification: if you see brlock, you know it's really
going to grab a lock.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit 9f295eecffd92e55584fc36539cd85cd32c832de)
commit 803035716338170896fee15f15b17c32e7ee777e
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:54:05 2010 +0930
tdb: rename tdb_release_extra_locks() to tdb_release_transaction_locks()
tdb_release_extra_locks() is too general: it carefully skips over the
transaction lock, even though the only caller then drops it. Change
this, and rename it to show it's clearly transaction-specific.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit a84222bbaf9ed2c7b9c61b8157b2e3c85f17fa32)
commit 74874ffb2c81e098c1d7935b37557c2151382ca6
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:54:05 2010 +0930
tdb: cleanup: remove ltype argument from _tdb_transaction_cancel.
Now the transaction allrecord lock is the standard one, and thus is cleaned
in tdb_release_extra_locks(), _tdb_transaction_cancel() doesn't need to
know what type it is.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit dd1b508c63034452673dbfee9956f52a1b6c90a5)
commit d7fdb38ac05b5f2af9eb485e98673280835273dd
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:54:05 2010 +0930
tdb: tdb_allrecord_lock/tdb_allrecord_unlock/tdb_allrecord_upgrade
Centralize locking of all chains of the tdb; rename _tdb_lockall to
tdb_allrecord_lock and _tdb_unlockall to tdb_allrecord_unlock, and
tdb_brlock_upgrade to tdb_allrecord_upgrade.
Then we use this in the transaction code. Unfortunately, if the transaction
code records that it has grabbed the allrecord lock read-only, write locks
will fail, so we treat this upgradable lock as a write lock, and mark it
as upgradable using the otherwise-unused offset field.
One subtlety: now the transaction code is using the allrecord_lock, the
tdb_release_extra_locks() function drops it for us, so we no longer need
to do it manually in _tdb_transaction_cancel.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit fca1621965c547e2d076eca2a2599e9629f91266)
commit 7a99cdf5d0a91764a750c1a264e90e5b66f910a1
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:54:05 2010 +0930
tdb: suppress record write locks when allrecord lock is taken.
Records themselves get (read) locked by the traversal code against delete.
Interestingly, this locking isn't done when the allrecord lock has been
taken, though the allrecord lock until recently didn't cover the actual
records (it now goes to end of file).
The write record lock, grabbed by the delete code, is not suppressed
by the allrecord lock. This is now bad: it causes us to punch a hole
in the allrecord lock when we release the write record lock. Make this
consistent: *no* record locks of any kind when the allrecord lock is
taken.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit caaf5c6baa1a4f340c1f38edd99b3a8b56621b8b)
commit 46f2c33357c999c31a8064c159c6162269c28d9d
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:54:05 2010 +0930
tdb: cleanup: always grab allrecord lock to infinity.
We were previously inconsistent with our "global" lock: the
transaction code grabbed it from FREELIST_TOP to end of file, and the
rest of the code grabbed it from FREELIST_TOP to end of the hash
chains. Change it to always grab to end of file for simplicity and
so we can merge the two.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit 9341f230f8968b4b18e451d15dda5ccbe7787768)
commit 587ac01ce836286aab54bfcb7a693a0170c7ebd3
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:54:05 2010 +0930
tdb: remove num_locks
This was redundant before this patch series: it mirrored num_lockrecs
exactly. It still does.
Also, skip useless branch when locks == 1: unconditional assignment is
cheaper anyway.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit 1ab8776247f89b143b6e58f4b038ab4bcea20d3a)
commit 2e8512403525c14c9b776ce28891d09c17ada91d
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:54:05 2010 +0930
tdb: use tdb_nest_lock() for seqnum lock.
This is pure overhead, but it centralizes the locking. Realloc (esp. as
most implementations are lazy) is fast compared to the fnctl anyway.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit d48c3e4982a38fb6b568ed3903e55e07a0fe5ca6)
commit 0a44584963232b0b1c62e30c9bede0439e68ef7d
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:53:51 2010 +0930
tdb: use tdb_nest_lock() for active lock.
Use our newly-generic nested lock tracking for the active lock.
Note that the tdb_have_extra_locks() and tdb_release_extra_locks()
functions have to skip over this lock now it is tracked.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit 4738d474c412cc59d26fcea64007e99094e8b675)
commit 6812d81907793299e874f121174d885f6500f374
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:53:51 2010 +0930
tdb: use tdb_nest_lock() for open lock.
This never nests, so it's overkill, but it centralizes the locking into
lock.c and removes the ugly flag in the transaction code to track whether
we have the lock or not.
Note that we have a temporary hack so this places a real lock, despite
the fact that we are in a transaction.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit 9136818df30c7179e1cffa18201cdfc990ebd7b7)
commit 4ca1b96a70048b2eaa0d12fb5f0fdb54ec396aa3
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:53:51 2010 +0930
tdb: use tdb_nest_lock() for transaction lock.
Rather than a boutique lock and a separate nest count, use our
newly-generic nested lock tracking for the transaction lock.
Note that the tdb_have_extra_locks() and tdb_release_extra_locks()
functions have to skip over this lock now it is tracked.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit e8fa70a321d489b454b07bd65e9b0d95084168de)
commit cfb154dd0f189f37b937e90144c2eb9e66a26420
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:53:51 2010 +0930
tdb: cleanup: find_nestlock() helper.
Factor out two loops which find locks; we are going to introduce a couple
more so a helper makes sense.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit ce41411c84760684ce539b6a302a0623a6a78a72)
commit d2dd720b51c4032e5d77d30212da8117d3f119df
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:53:51 2010 +0930
tdb: cleanup: tdb_release_extra_locks() helper
Move locking intelligence back into lock.c, rather than open-coding the
lock release in transaction.c.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit db270734d8b4208e00ce9de5af1af7ee11823f6d)
commit dda3587dfee598f387c2e696f3645486fac65052
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:53:51 2010 +0930
tdb: cleanup: tdb_have_extra_locks() helper
In many places we check whether locks are held: add a helper to do this.
The _tdb_lockall() case has already checked for the allrecord lock, so
the extra work done by tdb_have_extra_locks() is merely redundant.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit fba42f1fb4f81b8913cce5a23ca5350ba45f40e1)
commit 495f3554259b9dbf9ee673c4fe420d98e50e4901
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:53:51 2010 +0930
tdb: don't suppress the transaction lock because of the allrecord lock.
tdb_transaction_lock() and tdb_transaction_unlock() do nothing if we
hold the allrecord lock. However, the two locks don't overlap, so
this is wrong.
This simplification makes the transaction lock a straight-forward nested
lock.
There are two callers for these functions:
1) The transaction code, which already makes sure the allrecord_lock
isn't held.
2) The traverse code, which wants to stop transactions whether it has the
allrecord lock or not. There have been deadlocks here before, however
this should not bring them back (I hope!)
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit b754f61d235bdc3e410b60014d6be4072645e16f)
commit 28576ddbd9bf91049db8a4f9e9e7856ac5b8f48a
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:53:51 2010 +0930
tdb: cleanup: tdb_nest_lock/tdb_nest_unlock
Because fcntl locks don't nest, we track them in the tdb->lockrecs array
and only place/release them when the count goes to 1/0. We only do this
for record locks, so we simply place the list number (or -1 for the free
list) in the structure.
To generalize this:
1) Put the offset rather than list number in struct tdb_lock_type.
2) Rename _tdb_lock() to tdb_nest_lock, make it non-static and move the
allrecord check out to the callers (except the mark case which doesn't
care).
3) Rename _tdb_unlock() to tdb_nest_unlock(), make it non-static and
move the allrecord out to the callers (except mark again).
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit 5d9de604d92d227899e9b861c6beafb2e4fa61e0)
commit a912657fb50a78b9b328c4564281fb9f7f1b3766
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:53:51 2010 +0930
tdb: cleanup: rename global_lock to allrecord_lock.
The word global is overloaded in tdb. The global_lock inside struct
tdb_context is used to indicate we hold a lock across all the chains.
Rename it to allrecord_lock.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit e9114a758538d460d4f9deae5ce631bf44b1eff8)
commit a4f83910a485cf56f9b3df1dcf2ad36ebec57473
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:53:51 2010 +0930
tdb: cleanup: rename GLOBAL_LOCK to OPEN_LOCK.
The word global is overloaded in tdb. The GLOBAL_LOCK offset is used at
open time to serialize initialization (and by the transaction code to block
open).
Rename it to OPEN_LOCK.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit 7ab422d6fbd4f8be02838089a41f872d538ee7a7)
commit d728a7f65bcd5f1aedbee41d6db5c35f10cb417e
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:53:42 2010 +0930
tdb: make _tdb_transaction_cancel static.
Now tdb_open() calls tdb_transaction_cancel() instead of
_tdb_transaction_cancel, we can make it static.
Signed-off-by: Rusty Russell<rusty at rustcorp.com.au>
(Imported from commit a6e0ef87d25734760fe77b87a9fd11db56760955)
commit 7b5fdc9c588237c83a1e70e5437e2e5510055b92
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:53:42 2010 +0930
tdb: cleanup: split brlock and brunlock methods.
This is taken from the CCAN code base: rather than using tdb_brlock for
locking and unlocking, we split it into brlock and brunlock functions.
For extra debugging information, brunlock says what kind of lock it is
unlocking (even though fnctl locks don't need this). This requires an
extra argument to tdb_transaction_unlock() so we know whether the
lock was upgraded to a write lock or not.
We also use a "flags" argument tdb_brlock:
1) TDB_LOCK_NOWAIT replaces lck_type = F_SETLK (vs F_SETLKW).
2) TDB_LOCK_MARK_ONLY replaces setting TDB_MARK_LOCK bit in ltype.
3) TDB_LOCK_PROBE replaces the "probe" argument.
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit 452b4a5a6efeecfb5c83475f1375ddc25bcddfbe)
commit b0dff4ed35ab2423b8fcc801cdaaebaa0d7654bb
Author: Brad Hards <bradh at frogmouth.net>
Date: Thu Apr 22 13:53:42 2010 +0930
Spelling fixes for tdb.
Signed-off-by: Matthias Dieter Wallnöfer <mwallnoefer at yahoo.de>
(Imported from commit 09e756b1d651caef203a4b7e02234f6dea374b08)
commit e9ee4aaeb471a7f5ba4c97d3f76f406c1fe9b92f
Author: Andrew Tridgell <tridge at samba.org>
Date: Thu Apr 22 13:53:42 2010 +0930
tdb: use fdatasync() instead of fsync() in transactions
This might help on some filesystems
(Imported from commit 1373e748aa53fbd3afe4d2377208257d42628d86)
commit 8b0a8a96edf8611257e58ea81ed872dc03ca5da6
Author: Volker Lendecke <vl at samba.org>
Date: Thu Apr 22 13:53:42 2010 +0930
tdb: Apply some const, just for clarity
(Imported from commit 6824c6f46ba7c15e8af91d5aa8b21a946b63107b)
commit 8c8782ecbb347e026f67d82a39555c0e43b1e9f8
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:53:41 2010 +0930
tdb: fix recovery reuse after crash
If a process (or the machine) dies after just after writing the
recovery head (pointing at the end of file), the recovery record will filled
with 0x42. This will not invoke a recovery on open, since rec.magic
!= TDB_RECOVERY_MAGIC.
Unfortunately, the first transaction commit will happily reuse that
area: tdb_recovery_allocate() doesn't check the magic. The recovery
record has length 0x42424242, and it writes that back into the
now-valid-looking transaction header) for the next comer (which
happens to be tdb_wipe_all in my tests).
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit b37b452cb8c1f56b37b04abe7bffdede371ca361)
commit e0ca2e02120258aabca1e1586a58a8d672484fb5
Author: Rusty Russell <rusty at rustcorp.com.au>
Date: Thu Apr 22 13:53:26 2010 +0930
tdb: give a name to the invalid recovery area constant (0)
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
(Imported from commit 6269cdcd1538e2e3cead9e0f3c156b0363d607a0)
commit 8885b2206fba41ec289fda5dfd653ee676aa0dd3
Author: Simo Sorce <idra at samba.org>
Date: Thu Apr 22 13:53:21 2010 +0930
release-scripts: parametrize scripts
This should make it easier to keep all release scripts alined as it will reduce
the difference between them to ideally a few variables
Also moves the tdb script in the scripts directory.
(Imported from commit 6339de7f4fef46fb3ad32d1ecf9379f5b5d24ccb)
commit 84c971f33c24d32e5599aba7ba83bb474f7ac922
Author: Simo Sorce <idra at samba.org>
Date: Thu Apr 22 13:45:58 2010 +0930
tdb: raise version to 1.2.1
after recent fixes we need to raise the version to 1.2.1 so that
we can require also the right patched version.
(Imported from commit 70534adee10fc6f5bba2d9304668dc6508e5de5a)
-----------------------------------------------------------------------
Summary of changes:
lib/tdb/common/check.c | 2 +-
lib/tdb/common/dump.c | 4 +-
lib/tdb/common/error.c | 4 +-
lib/tdb/common/freelist.c | 10 +-
lib/tdb/common/io.c | 5 +-
lib/tdb/common/lock.c | 583 ++++++++++++++++++++++++++------------
lib/tdb/common/open.c | 42 ++--
lib/tdb/common/tdb.c | 35 ++-
lib/tdb/common/tdb_private.h | 49 +++-
lib/tdb/common/transaction.c | 166 ++++++------
lib/tdb/common/traverse.c | 18 +-
lib/tdb/configure.ac | 2 +-
lib/tdb/docs/README | 19 ++
lib/tdb/include/tdb.h | 133 +++++-----
lib/tdb/pytdb.c | 10 -
lib/tdb/python/tdbdump.py | 2 +-
lib/tdb/python/tests/simple.py | 2 +-
lib/tdb/release-script.sh | 48 ---
lib/tdb/script/release-script.sh | 67 +++++
lib/tdb/tdb.exports | 2 +-
lib/tdb/tdb.signatures | 1 +
lib/tdb/tools/tdbtorture.c | 199 +++++++++----
22 files changed, 883 insertions(+), 520 deletions(-)
delete mode 100755 lib/tdb/release-script.sh
create mode 100644 lib/tdb/script/release-script.sh
Changeset truncated at 500 lines:
diff --git a/lib/tdb/common/check.c b/lib/tdb/common/check.c
index f0a15f8..6bbfd7d 100644
--- a/lib/tdb/common/check.c
+++ b/lib/tdb/common/check.c
@@ -370,7 +370,7 @@ int tdb_check(struct tdb_context *tdb,
goto free;
break;
case TDB_RECOVERY_MAGIC:
- case 0: /* Used for invalid (or in-progress) recovery area. */
+ case TDB_RECOVERY_INVALID_MAGIC:
if (recovery_start != off) {
TDB_LOG((tdb, TDB_DEBUG_ERROR,
"Unexpected recovery record at offset %d\n",
diff --git a/lib/tdb/common/dump.c b/lib/tdb/common/dump.c
index bdcbfab..9f770f8 100644
--- a/lib/tdb/common/dump.c
+++ b/lib/tdb/common/dump.c
@@ -6,11 +6,11 @@
Copyright (C) Andrew Tridgell 1999-2005
Copyright (C) Paul `Rusty' Russell 2000
Copyright (C) Jeremy Allison 2000-2003
-
+
** NOTE! The following LGPL license applies to the tdb
** library. This does NOT imply that all of Samba is released
** under the LGPL
-
+
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
diff --git a/lib/tdb/common/error.c b/lib/tdb/common/error.c
index 195ab23..9197918 100644
--- a/lib/tdb/common/error.c
+++ b/lib/tdb/common/error.c
@@ -6,11 +6,11 @@
Copyright (C) Andrew Tridgell 1999-2005
Copyright (C) Paul `Rusty' Russell 2000
Copyright (C) Jeremy Allison 2000-2003
-
+
** NOTE! The following LGPL license applies to the tdb
** library. This does NOT imply that all of Samba is released
** under the LGPL
-
+
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
diff --git a/lib/tdb/common/freelist.c b/lib/tdb/common/freelist.c
index 8113b54..79e3c34 100644
--- a/lib/tdb/common/freelist.c
+++ b/lib/tdb/common/freelist.c
@@ -6,11 +6,11 @@
Copyright (C) Andrew Tridgell 1999-2005
Copyright (C) Paul `Rusty' Russell 2000
Copyright (C) Jeremy Allison 2000-2003
-
+
** NOTE! The following LGPL license applies to the tdb
** library. This does NOT imply that all of Samba is released
** under the LGPL
-
+
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
@@ -98,7 +98,7 @@ static int update_tailer(struct tdb_context *tdb, tdb_off_t offset,
}
/* Add an element into the freelist. Merge adjacent records if
- neccessary. */
+ necessary. */
int tdb_free(struct tdb_context *tdb, tdb_off_t offset, struct tdb_record *rec)
{
/* Allocation and tailer lock */
@@ -143,7 +143,7 @@ left:
tdb_off_t left = offset - sizeof(tdb_off_t);
struct tdb_record l;
tdb_off_t leftsize;
-
+
/* Read in tailer and jump back to header */
if (tdb_ofs_read(tdb, left, &leftsize) == -1) {
TDB_LOG((tdb, TDB_DEBUG_FATAL, "tdb_free: left offset read failed at %u\n", left));
@@ -334,7 +334,7 @@ tdb_off_t tdb_allocate(struct tdb_context *tdb, tdb_len_t length, struct tdb_rec
bestfit.rec_len < length * multiplier) {
break;
}
-
+
/* this multiplier means we only extremely rarely
search more than 50 or so records. At 50 records we
accept records up to 11 times larger than what we
diff --git a/lib/tdb/common/io.c b/lib/tdb/common/io.c
index d549715..058ca6c 100644
--- a/lib/tdb/common/io.c
+++ b/lib/tdb/common/io.c
@@ -6,11 +6,11 @@
Copyright (C) Andrew Tridgell 1999-2005
Copyright (C) Paul `Rusty' Russell 2000
Copyright (C) Jeremy Allison 2000-2003
-
+
** NOTE! The following LGPL license applies to the tdb
** library. This does NOT imply that all of Samba is released
** under the LGPL
-
+
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
@@ -461,7 +461,6 @@ static const struct tdb_methods io_methods = {
tdb_next_hash_chain,
tdb_oob,
tdb_expand_file,
- tdb_brlock
};
/*
diff --git a/lib/tdb/common/lock.c b/lib/tdb/common/lock.c
index 0984e51..285b7a3 100644
--- a/lib/tdb/common/lock.c
+++ b/lib/tdb/common/lock.c
@@ -6,11 +6,11 @@
Copyright (C) Andrew Tridgell 1999-2005
Copyright (C) Paul `Rusty' Russell 2000
Copyright (C) Jeremy Allison 2000-2003
-
+
** NOTE! The following LGPL license applies to the tdb
** library. This does NOT imply that all of Samba is released
** under the LGPL
-
+
This library is free software; you can redistribute it and/or
modify it under the terms of the GNU Lesser General Public
License as published by the Free Software Foundation; either
@@ -27,13 +27,104 @@
#include "tdb_private.h"
-#define TDB_MARK_LOCK 0x80000000
-
void tdb_setalarm_sigptr(struct tdb_context *tdb, volatile sig_atomic_t *ptr)
{
tdb->interrupt_sig_ptr = ptr;
}
+static int fcntl_lock(struct tdb_context *tdb,
+ int rw, off_t off, off_t len, bool waitflag)
+{
+ struct flock fl;
+
+ fl.l_type = rw;
+ fl.l_whence = SEEK_SET;
+ fl.l_start = off;
+ fl.l_len = len;
+ fl.l_pid = 0;
+
+ if (waitflag)
+ return fcntl(tdb->fd, F_SETLKW, &fl);
+ else
+ return fcntl(tdb->fd, F_SETLK, &fl);
+}
+
+static int fcntl_unlock(struct tdb_context *tdb, int rw, off_t off, off_t len)
+{
+ struct flock fl;
+#if 0 /* Check they matched up locks and unlocks correctly. */
+ char line[80];
+ FILE *locks;
+ bool found = false;
+
+ locks = fopen("/proc/locks", "r");
+
+ while (fgets(line, 80, locks)) {
+ char *p;
+ int type, start, l;
+
+ /* eg. 1: FLOCK ADVISORY WRITE 2440 08:01:2180826 0 EOF */
+ p = strchr(line, ':') + 1;
+ if (strncmp(p, " POSIX ADVISORY ", strlen(" POSIX ADVISORY ")))
+ continue;
+ p += strlen(" FLOCK ADVISORY ");
+ if (strncmp(p, "READ ", strlen("READ ")) == 0)
+ type = F_RDLCK;
+ else if (strncmp(p, "WRITE ", strlen("WRITE ")) == 0)
+ type = F_WRLCK;
+ else
+ abort();
+ p += 6;
+ if (atoi(p) != getpid())
+ continue;
+ p = strchr(strchr(p, ' ') + 1, ' ') + 1;
+ start = atoi(p);
+ p = strchr(p, ' ') + 1;
+ if (strncmp(p, "EOF", 3) == 0)
+ l = 0;
+ else
+ l = atoi(p) - start + 1;
+
+ if (off == start) {
+ if (len != l) {
+ fprintf(stderr, "Len %u should be %u: %s",
+ (int)len, l, line);
+ abort();
+ }
+ if (type != rw) {
+ fprintf(stderr, "Type %s wrong: %s",
+ rw == F_RDLCK ? "READ" : "WRITE", line);
+ abort();
+ }
+ found = true;
+ break;
+ }
+ }
+
+ if (!found) {
+ fprintf(stderr, "Unlock on %u@%u not found!\n",
+ (int)off, (int)len);
+ abort();
+ }
+
+ fclose(locks);
+#endif
+
+ fl.l_type = F_UNLCK;
+ fl.l_whence = SEEK_SET;
+ fl.l_start = off;
+ fl.l_len = len;
+ fl.l_pid = 0;
+
+ return fcntl(tdb->fd, F_SETLKW, &fl);
+}
+
+/* list -1 is the alloc list, otherwise a hash chain. */
+static tdb_off_t lock_offset(int list)
+{
+ return FREELIST_TOP + 4*list;
+}
+
/* a byte range locking function - return 0 on success
this functions locks/unlocks 1 byte at the specified offset.
@@ -42,30 +133,36 @@ void tdb_setalarm_sigptr(struct tdb_context *tdb, volatile sig_atomic_t *ptr)
note that a len of zero means lock to end of file
*/
-int tdb_brlock(struct tdb_context *tdb, tdb_off_t offset,
- int rw_type, int lck_type, int probe, size_t len)
+int tdb_brlock(struct tdb_context *tdb,
+ int rw_type, tdb_off_t offset, size_t len,
+ enum tdb_lock_flags flags)
{
- struct flock fl;
int ret;
if (tdb->flags & TDB_NOLOCK) {
return 0;
}
+ if (flags & TDB_LOCK_MARK_ONLY) {
+ return 0;
+ }
+
if ((rw_type == F_WRLCK) && (tdb->read_only || tdb->traverse_read)) {
tdb->ecode = TDB_ERR_RDONLY;
return -1;
}
- fl.l_type = rw_type;
- fl.l_whence = SEEK_SET;
- fl.l_start = offset;
- fl.l_len = len;
- fl.l_pid = 0;
+ /* Sanity check */
+ if (tdb->transaction && offset >= lock_offset(-1) && len != 0) {
+ tdb->ecode = TDB_ERR_RDONLY;
+ TDB_LOG((tdb, TDB_DEBUG_TRACE, "tdb_brlock attempted in transaction at offset %d rw_type=%d flags=%d len=%d\n",
+ offset, rw_type, flags, (int)len));
+ return -1;
+ }
do {
- ret = fcntl(tdb->fd,lck_type,&fl);
-
+ ret = fcntl_lock(tdb, rw_type, offset, len,
+ flags & TDB_LOCK_WAIT);
/* Check for a sigalarm break. */
if (ret == -1 && errno == EINTR &&
tdb->interrupt_sig_ptr &&
@@ -79,15 +176,34 @@ int tdb_brlock(struct tdb_context *tdb, tdb_off_t offset,
/* Generic lock error. errno set by fcntl.
* EAGAIN is an expected return from non-blocking
* locks. */
- if (!probe && lck_type != F_SETLK) {
- TDB_LOG((tdb, TDB_DEBUG_TRACE,"tdb_brlock failed (fd=%d) at offset %d rw_type=%d lck_type=%d len=%d\n",
- tdb->fd, offset, rw_type, lck_type, (int)len));
+ if (!(flags & TDB_LOCK_PROBE) && errno != EAGAIN) {
+ TDB_LOG((tdb, TDB_DEBUG_TRACE,"tdb_brlock failed (fd=%d) at offset %d rw_type=%d flags=%d len=%d\n",
+ tdb->fd, offset, rw_type, flags, (int)len));
}
return -1;
}
return 0;
}
+int tdb_brunlock(struct tdb_context *tdb,
+ int rw_type, tdb_off_t offset, size_t len)
+{
+ int ret;
+
+ if (tdb->flags & TDB_NOLOCK) {
+ return 0;
+ }
+
+ do {
+ ret = fcntl_unlock(tdb, rw_type, offset, len);
+ } while (ret == -1 && errno == EINTR);
+
+ if (ret == -1) {
+ TDB_LOG((tdb, TDB_DEBUG_TRACE,"tdb_brunlock failed (fd=%d) at offset %d rw_type=%d len=%d\n",
+ tdb->fd, offset, rw_type, (int)len));
+ }
+ return ret;
+}
/*
upgrade a read lock to a write lock. This needs to be handled in a
@@ -95,12 +211,29 @@ int tdb_brlock(struct tdb_context *tdb, tdb_off_t offset,
deadlock detection and claim a deadlock when progress can be
made. For those OSes we may loop for a while.
*/
-int tdb_brlock_upgrade(struct tdb_context *tdb, tdb_off_t offset, size_t len)
+int tdb_allrecord_upgrade(struct tdb_context *tdb)
{
int count = 1000;
+
+ if (tdb->allrecord_lock.count != 1) {
+ TDB_LOG((tdb, TDB_DEBUG_ERROR,
+ "tdb_allrecord_upgrade failed: count %u too high\n",
+ tdb->allrecord_lock.count));
+ return -1;
+ }
+
+ if (tdb->allrecord_lock.off != 1) {
+ TDB_LOG((tdb, TDB_DEBUG_ERROR,
+ "tdb_allrecord_upgrade failed: already upgraded?\n"));
+ return -1;
+ }
+
while (count--) {
struct timeval tv;
- if (tdb_brlock(tdb, offset, F_WRLCK, F_SETLKW, 1, len) == 0) {
+ if (tdb_brlock(tdb, F_WRLCK, FREELIST_TOP, 0,
+ TDB_LOCK_WAIT|TDB_LOCK_PROBE) == 0) {
+ tdb->allrecord_lock.ltype = F_WRLCK;
+ tdb->allrecord_lock.off = 0;
return 0;
}
if (errno != EDEADLK) {
@@ -111,57 +244,46 @@ int tdb_brlock_upgrade(struct tdb_context *tdb, tdb_off_t offset, size_t len)
tv.tv_usec = 1;
select(0, NULL, NULL, NULL, &tv);
}
- TDB_LOG((tdb, TDB_DEBUG_TRACE,"tdb_brlock_upgrade failed at offset %d\n", offset));
+ TDB_LOG((tdb, TDB_DEBUG_TRACE,"tdb_allrecord_upgrade failed\n"));
return -1;
}
-
-/* lock a list in the database. list -1 is the alloc list */
-static int _tdb_lock(struct tdb_context *tdb, int list, int ltype, int op)
+static struct tdb_lock_type *find_nestlock(struct tdb_context *tdb,
+ tdb_off_t offset)
{
- struct tdb_lock_type *new_lck;
- int i;
- bool mark_lock = ((ltype & TDB_MARK_LOCK) == TDB_MARK_LOCK);
+ unsigned int i;
- ltype &= ~TDB_MARK_LOCK;
-
- /* a global lock allows us to avoid per chain locks */
- if (tdb->global_lock.count &&
- (ltype == tdb->global_lock.ltype || ltype == F_RDLCK)) {
- return 0;
+ for (i=0; i<tdb->num_lockrecs; i++) {
+ if (tdb->lockrecs[i].off == offset) {
+ return &tdb->lockrecs[i];
+ }
}
+ return NULL;
+}
- if (tdb->global_lock.count) {
- tdb->ecode = TDB_ERR_LOCK;
- return -1;
- }
+/* lock an offset in the database. */
+int tdb_nest_lock(struct tdb_context *tdb, uint32_t offset, int ltype,
+ enum tdb_lock_flags flags)
+{
+ struct tdb_lock_type *new_lck;
- if (list < -1 || list >= (int)tdb->header.hash_size) {
+ if (offset >= lock_offset(tdb->header.hash_size)) {
tdb->ecode = TDB_ERR_LOCK;
- TDB_LOG((tdb, TDB_DEBUG_ERROR,"tdb_lock: invalid list %d for ltype=%d\n",
- list, ltype));
+ TDB_LOG((tdb, TDB_DEBUG_ERROR,"tdb_lock: invalid offset %u for ltype=%d\n",
+ offset, ltype));
return -1;
}
if (tdb->flags & TDB_NOLOCK)
return 0;
- for (i=0; i<tdb->num_lockrecs; i++) {
- if (tdb->lockrecs[i].list == list) {
- if (tdb->lockrecs[i].count == 0) {
- /*
- * Can't happen, see tdb_unlock(). It should
- * be an assert.
- */
- TDB_LOG((tdb, TDB_DEBUG_ERROR, "tdb_lock: "
- "lck->count == 0 for list %d", list));
- }
- /*
- * Just increment the in-memory struct, posix locks
- * don't stack.
- */
- tdb->lockrecs[i].count++;
- return 0;
- }
+ new_lck = find_nestlock(tdb, offset);
+ if (new_lck) {
+ /*
+ * Just increment the in-memory struct, posix locks
+ * don't stack.
+ */
+ new_lck->count++;
+ return 0;
}
new_lck = (struct tdb_lock_type *)realloc(
@@ -175,27 +297,89 @@ static int _tdb_lock(struct tdb_context *tdb, int list, int ltype, int op)
/* Since fcntl locks don't nest, we do a lock for the first one,
and simply bump the count for future ones */
- if (!mark_lock &&
- tdb->methods->tdb_brlock(tdb,FREELIST_TOP+4*list, ltype, op,
- 0, 1)) {
+ if (tdb_brlock(tdb, ltype, offset, 1, flags)) {
return -1;
}
- tdb->num_locks++;
-
- tdb->lockrecs[tdb->num_lockrecs].list = list;
+ tdb->lockrecs[tdb->num_lockrecs].off = offset;
tdb->lockrecs[tdb->num_lockrecs].count = 1;
tdb->lockrecs[tdb->num_lockrecs].ltype = ltype;
- tdb->num_lockrecs += 1;
+ tdb->num_lockrecs++;
return 0;
}
+static int tdb_lock_and_recover(struct tdb_context *tdb)
+{
+ int ret;
+
+ /* We need to match locking order in transaction commit. */
+ if (tdb_brlock(tdb, F_WRLCK, FREELIST_TOP, 0, TDB_LOCK_WAIT)) {
+ return -1;
+ }
+
+ if (tdb_brlock(tdb, F_WRLCK, OPEN_LOCK, 1, TDB_LOCK_WAIT)) {
+ tdb_brunlock(tdb, F_WRLCK, FREELIST_TOP, 0);
+ return -1;
+ }
+
+ ret = tdb_transaction_recover(tdb);
+
+ tdb_brunlock(tdb, F_WRLCK, OPEN_LOCK, 1);
+ tdb_brunlock(tdb, F_WRLCK, FREELIST_TOP, 0);
+
+ return ret;
+}
+
+static bool have_data_locks(const struct tdb_context *tdb)
+{
+ unsigned int i;
+
+ for (i = 0; i < tdb->num_lockrecs; i++) {
+ if (tdb->lockrecs[i].off >= lock_offset(-1))
+ return true;
+ }
+ return false;
--
CTDB repository
More information about the samba-cvs
mailing list