[SCM] Samba Shared Repository - branch v3-5-test updated
Karolin Seeger
kseeger at samba.org
Wed Mar 31 09:56:34 MDT 2010
The branch, v3-5-test has been updated
via f3ade09... s3: Remove the unused parameter "persistent" from fetch_locked_internal (cherry picked from commit a5db27936e9c6aad99300ea46808481803f57e08)
via a3b71fc... s3: db->persistent==true was handled earlier, make this more obvious (cherry picked from commit c7835a4845bbc7e4d340a75229866b2d4946f6eb)
via 5423a2c... s3: Add "g_lock_do" as a convenience wrapper function (cherry picked from commit 79100c242153ea174a4405afd45cbf635da313aa)
via f73b306... s3: Fix timeout calculation if g_lock_lock is given a timeout < 60s
via eb15655... s3: Slightly increase parallelism in g_lock
via c5babce... s3: Avoid starving locks when many processes die at the same time
via 7e787c5... s3: Avoid a thundering herd in g_lock_unlock
via 9673c93... s3: Optimize g_lock_lock for a heavily contended case
via b2677d8... s3: Fix handling of processes that died in g_lock
via 8d4a14f... s3: Fix a typo (cherry picked from commit bac235dd302570850bb25194ff4bd39b6d653f0d)
via d0ce707... Fix warning messages on compile in g_lock.c Volker & Michael please check.
via 0a030b0... s3:g_lock: remove a nested event loop, replacing the inner loop by select
via 84505cb... s3:ctdb_conn: add ctdbd_conn_get_fd() to get the fd out of the ctdb connection
via 77e4a72... s3:g_lock: remove an unreached code path.
via c17dace... s3:dbwrap_ctdb: fix reading/storing of special key __db_sequence_number__
via 6f325fb... s3:dbwrap_ctdb: exit early when nothing has been written in transaction_commit.
via 7fc2b61... s3:dbwrap_ctdb: fix brown paperbag bug in ctdb_transaction_commit.
via e5cc0e7... s3:dbwrap_ctdb: fix logic error in pull_newest_from_marshall_buffer().
via 89cb10d... s3:dbwrap_ctdb: fix an uninitialized variable.
via dfbc45f... s3:dbwrap_ctdb: fix two "may be used uninitialized" warnings
via 8926082... s3:dbwrap_ctdb: fix db_ctdb_fetch_db_seqnum_from_db() when NT_STATUS_NOT_FOUND.
via ffaa072... s3:build: remove checks for deprecated ctdb controls.
via cfdf2d7... s3:dbwrap_ctdb: maintain a database sequence number that bumps in transactions
via 06943fe... s3:dbwrap_ctdb: change db_ctdb_transaction_store() to return NTSTATUS.
via 60aabc9... s3:dbwrap_ctdb: update (C)
via ea3c4c9... build: Add a configure check for CTDB_CONTROL_TRANS3_COMMIT.
via c2354aa... s3:dbwrap_ctdb: start rewrite of transactions using the global lock (g_lock)
via 3f64cba... s3: Add tdb_data_equal (cherry picked from commit ebc08b9938a4d266be16ca7e06d27813952cd00f)
via dcfbe61... s3:torture: add a test LOCAL-DBTRANS to torture dbwrap with transactions.
via 139a9fd... s3: setup debug for smbtorture (cherry picked from commit b13dd17840a598ae3441e48b130a2b2a2b940572)
via 803cf6d... s3: Add ctdb_conn_msg_ctx() (cherry picked from commit 12abab711b58237ddccfa1d9bb526f8c7dbb6e9f)
via 98873e0... s3: Implement global locks in a g_lock tdb
via 675c4b5... s3-smbd: add a rate limited cleanup of brl, connections and locking db
via 6a78929... s3-brlock: we don't need these MSG_SMB_UNLOCK calls now
via aa9ab3e... s3-brlock: add a minimim retry time for pending blocking locks
via 380e86e... s3-events: make the old timed events compatible with tevent
from 9620f5f... Fix bug 7307 - man net usershare mistake
http://gitweb.samba.org/?p=samba.git;a=shortlog;h=v3-5-test
- Log -----------------------------------------------------------------
commit f3ade095089d242484afd0f7fdbcc7826c311368
Author: Volker Lendecke <vl at samba.org>
Date: Fri Mar 5 15:30:22 2010 +0100
s3: Remove the unused parameter "persistent" from fetch_locked_internal (cherry picked from commit a5db27936e9c6aad99300ea46808481803f57e08)
The last 32 patches address bug #7232 (CTDB persistent transactions are racy).
commit a3b71fc3b903d23e0866d7c7779a96dc81450dca
Author: Volker Lendecke <vl at samba.org>
Date: Fri Mar 5 15:28:39 2010 +0100
s3: db->persistent==true was handled earlier, make this more obvious (cherry picked from commit c7835a4845bbc7e4d340a75229866b2d4946f6eb)
commit 5423a2cc8138db852fefa133a7da53dc5abf1961
Author: Volker Lendecke <vl at samba.org>
Date: Fri Mar 12 14:22:54 2010 +0100
s3: Add "g_lock_do" as a convenience wrapper function (cherry picked from commit 79100c242153ea174a4405afd45cbf635da313aa)
commit f73b306797f354d0a24ff8af40b05d6151e52a06
Author: Volker Lendecke <vl at samba.org>
Date: Tue Feb 16 15:21:25 2010 +0100
s3: Fix timeout calculation if g_lock_lock is given a timeout < 60s
Detected while showing this code to obnox :-)
(cherry picked from commit f8b246e44c819b909b23b4b98ef0999c84d2f4ff)
commit eb15655fe1fb71fb7145ce9f479cc112a459f72e
Author: Volker Lendecke <vl at samba.org>
Date: Tue Feb 16 12:31:58 2010 +0100
s3: Slightly increase parallelism in g_lock
There's no need to still hold the g_lock tdb-level lock while telling the
waiters to retry
(cherry picked from commit 83542d973ca771353109c7da4b0391d6ba910f53)
commit c5babceaa95fd163f785e1dfe5521b93402e06a3
Author: Volker Lendecke <vl at samba.org>
Date: Tue Feb 16 12:28:53 2010 +0100
s3: Avoid starving locks when many processes die at the same time
In g_lock_unlock we have a little race between the process_exists and
messaging_send call: We only send to 5 waiters now, they all might have died
between us checking their existence and sending the message. This change makes
g_lock_lock retry at least once every minute.
(cherry picked from commit be919d6faed198cdc29322a4d9491946c0b044b3)
commit 7e787c513e47875e7dbfc195d8bd1cecec3f08ce
Author: Volker Lendecke <vl at samba.org>
Date: Tue Feb 16 12:22:08 2010 +0100
s3: Avoid a thundering herd in g_lock_unlock
Only notify the first 5 pending lock waiters. This avoids a thundering herd
problem that is really nasty in a cluster. It also makes acquiring a lock a bit
more FIFO, lock waiters are added to the end of the array.
(cherry picked from commit 725b3654f831fbe0388cc09f46269903c9eef1d7)
commit 9673c93933624ae6e02e83dae3c19351edfd301a
Author: Volker Lendecke <vl at samba.org>
Date: Mon Feb 15 16:57:16 2010 +0100
s3: Optimize g_lock_lock for a heavily contended case
Only check the existence of the lock owner in g_lock_parse, check the rest of
the records only when we got the lock successfully. This reduces the load on
process_exists which can involve a network roundtrip in the clustered case.
(cherry picked from commit 07978bd175395e0dc770f68fff5b8bd8b0fdeb51)
commit b2677d82dc5c6bc76b0343d90ec75972b5b2a8bf
Author: Volker Lendecke <vl at samba.org>
Date: Mon Feb 15 16:49:46 2010 +0100
s3: Fix handling of processes that died in g_lock
g_lock_parse might have thrown away entries from the locks array because the
processes were not around anymore. Don't store the orphaned entries.
(cherry picked from commit f3bdb163f461175c50b4930fa3464beaee30f4a8)
commit 8d4a14f3aa04812cd81aa4d56b878169dc41899c
Author: Volker Lendecke <vl at samba.org>
Date: Mon Feb 15 16:35:06 2010 +0100
s3: Fix a typo (cherry picked from commit bac235dd302570850bb25194ff4bd39b6d653f0d)
commit d0ce7072bd89d11bb63b1fa976305d7087dcb9af
Author: Jeremy Allison <jra at samba.org>
Date: Fri Feb 12 22:21:19 2010 -0800
Fix warning messages on compile in g_lock.c Volker & Michael please check.
Jeremy.
(cherry picked from commit 10e54fb422d9f1ae6d33e5fabbf8c651b0e57a8c)
commit 0a030b0de66e84f35637699377e44a083c445fa2
Author: Michael Adam <obnox at samba.org>
Date: Sat Jan 23 01:17:06 2010 +0100
s3:g_lock: remove a nested event loop, replacing the inner loop by select
This made smbd crash in g_lock_lock() when trying to start a
transaction on a db with an already started transaction,
e.g. in a tcon_and_X where the share_info.tdb was not yet
initialized but share_info.tdb was already locked by another
process or writing acces to the winreg rpc pipe where the
registry tdb was already locked by another process.
What we really _want_ to do here by design is to react to
MSG_DBWRAP_G_LOCK_RETRY messages that are either sent
by a client doing g_lock_unlock or by ourselves when
we receive a CTDB_SRVID_SAMBA_NOTIFY or
CTDB_SRVID_RECONFIGURE message from ctdbd, i.e. when
either a client holding a lock or a complete node
has died.
Doing this properly involves calling tevent_loop_once(),
but doing this here with the main ctdbd messaging context
creates a nested event loop when g_lock_lock() is called
from the main event loop.
So as a quick fix, we act a little corasely here: we do
a select on the ctdb connection fd and when it is readable
or we get EINTR, then we retry without actually parsing
any ctdb packages or dispatching messages. This means that
we retry more often than necessary and intended by design,
but this does not harm and it is unobtrusive. When we have
finished, the main loop will pick up all the messages and
ctdb packets. The only extra twist is that we cannot use
timed events here but have to handcode a timeout for select.
Michael
(cherry picked from commit 83fffbeb44441a87569e543054af21d975eb20ae)
commit 84505cb3157d23b6df170bf694cd6adecc1fe9f5
Author: Michael Adam <obnox at samba.org>
Date: Sat Jan 23 00:05:15 2010 +0100
s3:ctdb_conn: add ctdbd_conn_get_fd() to get the fd out of the ctdb connection
Michael
(cherry picked from commit e4af0bc5af2c3ee025ca7fac251c3672ba2c8dd5)
commit 77e4a72902856cd6c1f635dc3a952f846522b7d0
Author: Michael Adam <obnox at samba.org>
Date: Fri Jan 22 15:56:28 2010 +0100
s3:g_lock: remove an unreached code path.
Michael
(cherry picked from commit 8e306b51b79d3dacd68be9f13aa8455e2eb4c03f)
commit c17daced42b3ea78fa8e6f6afc8c2550cac53ca0
Author: Michael Adam <obnox at samba.org>
Date: Mon Jan 18 17:26:04 2010 +0100
s3:dbwrap_ctdb: fix reading/storing of special key __db_sequence_number__
The key for reading and writing was inconsistent due to a
off by one data length.
Michael
(cherry picked from commit 1933214108d1a71bc6473a696ce35020a427d8f4)
commit 6f325fb6b092da554180218d1cbb42181f90bbfa
Author: Michael Adam <obnox at samba.org>
Date: Wed Jan 13 23:53:54 2010 +0100
s3:dbwrap_ctdb: exit early when nothing has been written in transaction_commit.
This skips update of the __db_sequence_number__ record when nothing else has
been written. There are transactions that are just openend and then nothing
is written until transaction_commit is called. This is for instance the case
with registry initialization routines: They start a transaction and only
write somthing when the registry has not been initialized yet.
So this change will skip many db_seqnum bumps and TRANS3_COMMIT roundtrips.
Michael
(cherry picked from commit c311697aded87ce624d40cbf14e05d6e6377c257)
commit 7fc2b612b8f77751effadcd53800c96e83e76418
Author: Michael Adam <obnox at samba.org>
Date: Wed Jan 13 23:51:34 2010 +0100
s3:dbwrap_ctdb: fix brown paperbag bug in ctdb_transaction_commit.
I carefully prepared the return value only to "return 0;" at the bottom. :-(
This may well have hit us for instance in the nested cancel case
and produced random errors.
Michael
(cherry picked from commit 1d594bd734a2f7146ed52872456a16c5e41816f1)
commit e5cc0e718863d6cfaa59fbb4cac7d9b70fba0381
Author: Michael Adam <obnox at samba.org>
Date: Wed Jan 6 00:37:21 2010 +0100
s3:dbwrap_ctdb: fix logic error in pull_newest_from_marshall_buffer().
The logic bug was that if a record was found in the marshall buffer,
then always the ctdb header of tha last record in the marshall buffer
was returned, and not the ctdb header of the last occurrence of the
requested record.
This is fixed by introducing an additional temporary variable.
Michael
(cherry picked from commit 524072b56bf659002410a817749bf86fe6f51e83)
commit 89cb10de5ec46384c5ae8d558ac29e4f9e00e8a1
Author: Michael Adam <obnox at samba.org>
Date: Tue Jan 5 16:17:27 2010 +0100
s3:dbwrap_ctdb: fix an uninitialized variable.
Michael
(cherry picked from commit 1505b69dea6044a13a59f672e22f5833256cb981)
commit dfbc45f6d6ef2fb99652992e291b1bf5276dec28
Author: Michael Adam <obnox at samba.org>
Date: Sat Dec 12 00:38:14 2009 +0100
s3:dbwrap_ctdb: fix two "may be used uninitialized" warnings
Michael
(cherry picked from commit fb981cdb8282d3b9b46d9ca515a5685add232a72)
commit 8926082d096b2a6e8688b1668bd7293632ebb508
Author: Michael Adam <obnox at samba.org>
Date: Sat Dec 12 00:30:37 2009 +0100
s3:dbwrap_ctdb: fix db_ctdb_fetch_db_seqnum_from_db() when NT_STATUS_NOT_FOUND.
Don't treat this as an error but return seqnum 0 instead.
Michael
(cherry picked from commit 10a44ee6930bb51b4b20ce42f35bc455ac1b7293)
commit ffaa072801d94ffcac837e77ff9e6538ba335150
Author: Michael Adam <obnox at samba.org>
Date: Fri Dec 11 16:45:38 2009 +0100
s3:build: remove checks for deprecated ctdb controls.
Michael
(cherry picked from commit 9113ce82b59c718ac709eb01b125e9e6746a96b7)
commit cfdf2d76878ee1ba32b2cb89f9ed740386990530
Author: Michael Adam <obnox at samba.org>
Date: Fri Dec 11 14:07:28 2009 +0100
s3:dbwrap_ctdb: maintain a database sequence number that bumps in transactions
For persistent databases, 64bit integer is kept in a special record
__db_sequence_number__. This record is incremented with each completed
transaction.
The retry mechanism for failing TRANS3_COMMIT controls inside the
db_ctdb_transaction_commit() function now relies one a modified
behaviour of ctdbd's treatment of persistent databases in recoveries.
Recently, a special treatment for persistent databases had been
introduced in ctdb (1.0.108) to work around the problems with the
orinal design of persistent transactions.
Now with the rewrite we need to revert to the old behaviour that
ctdb always takes the newest copies of all records.
This change also paves the way for a next step, which will make
recovery use the db seqnum to tell which node has the newest copy
of a persistent db and use that node's copy. This will greatly
reduce the amount of data transferred with each recovery.
Michael
(cherry picked from commit 3fe7ce141d6afe3825b06c5feb90558911e4df1e)
commit 06943fe1e6982c9df4886ee0a17572e502726897
Author: Michael Adam <obnox at samba.org>
Date: Fri Dec 11 12:30:57 2009 +0100
s3:dbwrap_ctdb: change db_ctdb_transaction_store() to return NTSTATUS.
The return values calculated by the callers were wrong anyways since
the new marshalling code does not set the local tdbs tdb error code.
Michael
(cherry picked from commit 26225d3e798892b39b3c238b0bee465bffac6550)
commit 60aabc946ed981efe7e37bb2b39307e82dd20fcc
Author: Michael Adam <obnox at samba.org>
Date: Fri Dec 11 10:35:50 2009 +0100
s3:dbwrap_ctdb: update (C)
Michael
(cherry picked from commit 5a0c42770b349877928a2b3fd8316903dd62e5b7)
commit ea3c4c96e0fd1f2165282589becb12bfb40700d3
Author: Michael Adam <obnox at samba.org>
Date: Fri Dec 4 11:49:21 2009 +0100
build: Add a configure check for CTDB_CONTROL_TRANS3_COMMIT.
This is the new implementation of ctdb transactions using the
global lock feature. It is needed by the current dbwrap_ctdb code.
Michael
(cherry picked from commit d4c0afa841ecdae1cab955cc73360deae23f5873)
commit c2354aa193617ffc3e4036fe0b9034c2664901fd
Author: Michael Adam <obnox at samba.org>
Date: Thu Dec 3 17:29:54 2009 +0100
s3:dbwrap_ctdb: start rewrite of transactions using the global lock (g_lock)
This simplifies the transaction code a lot:
* transaction_start essentially consists of acquiring a global lock.
* No write operations at all are performed on the local database
until the transaction is committed: Every store operation is just
going into the marshall buffer.
* The commit operation calls a new simplified TRANS3_COMMIT control
in ctdb which rolls out thae changes to all nodes including the
node that is performing the transaction.
Michael
(cherry picked from commit 16bc6ba2268e3660d026076264de8666356e00bf)
commit 3f64cba12aea87ad57bd52547f7dc312cd9e8636
Author: Volker Lendecke <vl at samba.org>
Date: Thu Dec 3 18:43:49 2009 +0100
s3: Add tdb_data_equal (cherry picked from commit ebc08b9938a4d266be16ca7e06d27813952cd00f)
commit dcfbe61ab0381491e19827806cb06e0822fa69e7
Author: Volker Lendecke <vl at samba.org>
Date: Fri Dec 11 15:37:52 2009 +0100
s3:torture: add a test LOCAL-DBTRANS to torture dbwrap with transactions.
commit 139a9fdcd6c33ef10af83fd9c640ed0191670c5b
Author: Volker Lendecke <vl at samba.org>
Date: Mon Dec 7 00:36:51 2009 +0100
s3: setup debug for smbtorture (cherry picked from commit b13dd17840a598ae3441e48b130a2b2a2b940572)
commit 803cf6da0bb93760902e6118952d73967d6fcdff
Author: Volker Lendecke <vl at samba.org>
Date: Fri Dec 4 13:22:30 2009 +0100
s3: Add ctdb_conn_msg_ctx() (cherry picked from commit 12abab711b58237ddccfa1d9bb526f8c7dbb6e9f)
commit 98873e0d6a38abdcaed48af6504a9a39a23ae027
Author: Volker Lendecke <vl at samba.org>
Date: Sun Oct 25 16:12:12 2009 +0100
s3: Implement global locks in a g_lock tdb
This is the basis to implement global locks in ctdb without depending on a
shared file system. The initial goal is to make ctdb persistent transactions
deterministic without too many timeouts.
(cherry picked from commit 4c1c3f2549f32fd069e0e7bf3aec299213f1e85b)
commit 675c4b5522394e4431a88d9ab7647ca24765feb6
Author: Andrew Tridgell <tridge at samba.org>
Date: Fri Feb 5 21:08:56 2010 -0800
s3-smbd: add a rate limited cleanup of brl, connections and locking db
On unclean shutdown we can end up with stale entries in the brlock,
connections and locking db. Previously we would do the cleanup on
every unclean exit, but that can cause smbd to be completely
unavailable for several minutes when a large number of child smbd
processes exit.
This adds a rate limited cleanup of the databases, with the default
that cleanup happens at most every 20s
(cherry picked from commit dd498d2eecf124a03b6117ddab892a1112f9e9db)
The last 4 patches address bug #7312 (Many disconnecting clients renders
clustered samba unusuable for some time).
commit 6a78929fd0543e027912346c30e6208b4a1f4362
Author: Andrew Tridgell <tridge at samba.org>
Date: Fri Feb 5 21:02:24 2010 -0800
s3-brlock: we don't need these MSG_SMB_UNLOCK calls now
These have been replaced with the min timeout in blocking.c
(cherry picked from commit 74267d652485cdcb711f734f0d80da0fb1495867)
commit aa9ab3ea57808b8e227c960b442911219405f1bf
Author: Andrew Tridgell <tridge at samba.org>
Date: Fri Feb 5 20:59:43 2010 -0800
s3-brlock: add a minimim retry time for pending blocking locks
When we are waiting on a pending byte range lock, another smbd might
exit uncleanly, and therefore not notify us of the removal of the
lock, and thus not trigger the lock to be retried.
We coped with this up to now by adding a message_send_all() in the
SIGCHLD and cluster reconfigure handlers to send a MSG_SMB_UNLOCK to
all smbd processes. That would generate O(N^2) work when a large
number of clients disconnected at once (such as on a network outage),
which could leave the whole system unusable for a very long time (many
minutes, or even longer).
By adding a minimum re-check time for pending byte range locks we
avoid this problem by ensuring that pending locks are retried at a
more regular interval.
(cherry picked from commit 5b398edbee672392f2cea260ab17445ecca927d7)
commit 380e86e0f9c9314f4f69528227668dbbb222b144
Author: Andrew Tridgell <tridge at samba.org>
Date: Fri Feb 5 19:14:45 2010 -0800
s3-events: make the old timed events compatible with tevent
tevent ensures that a timed event is only called once. The old events
code relied on the called handler removing the event itself. If the
handler removed the event after calling a function which invoked the
event loop then the timed event could loop forever.
This change makes the two timed event systems more compatible, by
allowing the handler to free the te if it wants to, but ensuring it is
off the linked list of events before the handler is called, and
ensuring it is freed even if the handler doesn't free it.
(cherry picked from commit 5dbf175c75bd6139f3238f36665000641f7f7f79)
-----------------------------------------------------------------------
Summary of changes:
lib/util/util_tdb.c | 8 +
lib/util/util_tdb.h | 1 +
source3/Makefile.in | 2 +
source3/configure.in | 23 +-
source3/include/ctdbd_conn.h | 5 +
source3/include/g_lock.h | 59 +++
source3/lib/ctdbd_conn.c | 89 ++++-
source3/lib/dbwrap_ctdb.c | 611 ++++++++++++--------------
source3/lib/events.c | 21 +-
source3/lib/g_lock.c | 766 ++++++++++++++++++++++++++++++++
source3/librpc/gen_ndr/messaging.h | 4 +-
source3/librpc/gen_ndr/ndr_messaging.c | 1 +
source3/librpc/idl/messaging.idl | 3 +-
source3/smbd/blocking.c | 20 +
source3/smbd/server.c | 50 ++-
source3/torture/torture.c | 132 ++++++
source3/utils/net.c | 7 +
source3/utils/net_g_lock.c | 208 +++++++++
source3/utils/net_proto.h | 3 +
19 files changed, 1640 insertions(+), 373 deletions(-)
create mode 100644 source3/include/g_lock.h
create mode 100644 source3/lib/g_lock.c
create mode 100644 source3/utils/net_g_lock.c
Changeset truncated at 500 lines:
diff --git a/lib/util/util_tdb.c b/lib/util/util_tdb.c
index cda8dc7..46dbf6d 100644
--- a/lib/util/util_tdb.c
+++ b/lib/util/util_tdb.c
@@ -38,6 +38,14 @@ TDB_DATA make_tdb_data(const uint8_t *dptr, size_t dsize)
return ret;
}
+bool tdb_data_equal(TDB_DATA t1, TDB_DATA t2)
+{
+ if (t1.dsize != t2.dsize) {
+ return false;
+ }
+ return (memcmp(t1.dptr, t2.dptr, t1.dsize) == 0);
+}
+
TDB_DATA string_tdb_data(const char *string)
{
return make_tdb_data((const uint8_t *)string, string ? strlen(string) : 0 );
diff --git a/lib/util/util_tdb.h b/lib/util/util_tdb.h
index da6378e..79c4671 100644
--- a/lib/util/util_tdb.h
+++ b/lib/util/util_tdb.h
@@ -6,6 +6,7 @@
Make a TDB_DATA and keep the const warning in one place
****************************************************************/
TDB_DATA make_tdb_data(const uint8_t *dptr, size_t dsize);
+bool tdb_data_equal(TDB_DATA t1, TDB_DATA t2);
TDB_DATA string_tdb_data(const char *string);
TDB_DATA string_term_tdb_data(const char *string);
diff --git a/source3/Makefile.in b/source3/Makefile.in
index f84ed20..f58bb70 100644
--- a/source3/Makefile.in
+++ b/source3/Makefile.in
@@ -265,6 +265,7 @@ EXTRA_ALL_TARGETS = @EXTRA_ALL_TARGETS@
TDB_LIB_OBJ = lib/util_tdb.o ../lib/util/util_tdb.o \
lib/dbwrap.o lib/dbwrap_tdb.o \
lib/dbwrap_ctdb.o \
+ lib/g_lock.o \
lib/dbwrap_rbt.o
TDB_VALIDATE_OBJ = lib/tdb_validate.o
@@ -1011,6 +1012,7 @@ NET_OBJ1 = utils/net.o utils/net_ads.o utils/net_help.o \
utils/net_conf.o utils/net_join.o utils/net_user.o \
utils/net_group.o utils/net_file.o utils/net_registry.o \
auth/token_util.o utils/net_dom.o utils/net_share.o \
+ utils/net_g_lock.o \
utils/net_eventlog.o
# these are not processed by make proto
diff --git a/source3/configure.in b/source3/configure.in
index a9b9e47..3111d5a 100644
--- a/source3/configure.in
+++ b/source3/configure.in
@@ -5233,7 +5233,7 @@ AC_CHECK_HEADERS(ctdb.h ctdb_private.h,,,[
#include <ctdb.h>
])
-AC_HAVE_DECL(CTDB_CONTROL_TRANS2_COMMIT_RETRY,[
+AC_HAVE_DECL(CTDB_CONTROL_TRANS3_COMMIT,[
#include "confdefs.h"
#define NO_CONFIG_H
#include "replace.h"
@@ -5244,27 +5244,10 @@ AC_HAVE_DECL(CTDB_CONTROL_TRANS2_COMMIT_RETRY,[
#include <ctdb.h>
#include <ctdb_private.h>
])
-if test x"$ac_cv_have_CTDB_CONTROL_TRANS2_COMMIT_RETRY_decl" = x"yes"; then
+if test x"$ac_cv_have_CTDB_CONTROL_TRANS3_COMMIT_decl" = x"yes"; then
ctdb_broken=no
else
- ctdb_broken="missing transaction support"
-fi
-
-AC_HAVE_DECL(CTDB_CONTROL_TRANS2_ACTIVE,[
-#include "confdefs.h"
-#define NO_CONFIG_H
-#include "replace.h"
-#include "system/wait.h"
-#include "system/network.h"
-#include <talloc.h>
-#include <tdb.h>
-#include <ctdb.h>
-#include <ctdb_private.h>
-])
-if test x"$ac_cv_have_CTDB_CONTROL_TRANS2_ACTIVE_decl" = x"yes"; then
- ctdb_broken=no
-else
- ctdb_broken="transaction support too old"
+ ctdb_broken="ctdb transaction support missing or too old"
fi
# in ctdb 1.0.57 ctdb_control_tcp was temparary renamed to ctdb_tcp_client
diff --git a/source3/include/ctdbd_conn.h b/source3/include/ctdbd_conn.h
index d721235..c5ba572 100644
--- a/source3/include/ctdbd_conn.h
+++ b/source3/include/ctdbd_conn.h
@@ -31,6 +31,9 @@ uint32 ctdbd_vnn(const struct ctdbd_connection *conn);
NTSTATUS ctdbd_register_msg_ctx(struct ctdbd_connection *conn,
struct messaging_context *msg_ctx);
+struct messaging_context *ctdb_conn_msg_ctx(struct ctdbd_connection *conn);
+
+int ctdbd_conn_get_fd(struct ctdbd_connection *conn);
NTSTATUS ctdbd_messaging_send(struct ctdbd_connection *conn,
uint32 dst_vnn, uint64 dst_srvid,
@@ -73,5 +76,7 @@ NTSTATUS ctdbd_control_local(struct ctdbd_connection *conn, uint32 opcode,
uint64_t srvid, uint32_t flags, TDB_DATA data,
TALLOC_CTX *mem_ctx, TDB_DATA *outdata,
int *cstatus);
+NTSTATUS ctdb_watch_us(struct ctdbd_connection *conn);
+NTSTATUS ctdb_unwatch(struct ctdbd_connection *conn);
#endif /* _CTDBD_CONN_H */
diff --git a/source3/include/g_lock.h b/source3/include/g_lock.h
new file mode 100644
index 0000000..b2d2c2d
--- /dev/null
+++ b/source3/include/g_lock.h
@@ -0,0 +1,59 @@
+/*
+ Unix SMB/CIFS implementation.
+ global locks based on ctdb
+ Copyright (C) 2009 by Volker Lendecke
+
+ This program is free software; you can redistribute it and/or modify
+ it under the terms of the GNU General Public License as published by
+ the Free Software Foundation; either version 3 of the License, or
+ (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU General Public License for more details.
+
+ You should have received a copy of the GNU General Public License
+ along with this program. If not, see <http://www.gnu.org/licenses/>.
+*/
+
+#ifndef _G_LOCK_H_
+#define _G_LOCK_H_
+
+#include "dbwrap.h"
+
+struct g_lock_ctx;
+
+enum g_lock_type {
+ G_LOCK_READ = 0,
+ G_LOCK_WRITE = 1,
+};
+
+/*
+ * Or'ed with g_lock_type
+ */
+#define G_LOCK_PENDING (2)
+
+struct g_lock_ctx *g_lock_ctx_init(TALLOC_CTX *mem_ctx,
+ struct messaging_context *msg);
+
+NTSTATUS g_lock_lock(struct g_lock_ctx *ctx, const char *name,
+ enum g_lock_type lock_type, struct timeval timeout);
+NTSTATUS g_lock_unlock(struct g_lock_ctx *ctx, const char *name);
+NTSTATUS g_lock_get(struct g_lock_ctx *ctx, const char *name,
+ struct server_id *pid);
+
+NTSTATUS g_lock_do(const char *name, enum g_lock_type lock_type,
+ struct timeval timeout,
+ void (*fn)(void *private_data), void *private_data);
+
+int g_lock_locks(struct g_lock_ctx *ctx,
+ int (*fn)(const char *name, void *private_data),
+ void *private_data);
+NTSTATUS g_lock_dump(struct g_lock_ctx *ctx, const char *name,
+ int (*fn)(struct server_id pid,
+ enum g_lock_type lock_type,
+ void *private_data),
+ void *private_data);
+
+#endif
diff --git a/source3/lib/ctdbd_conn.c b/source3/lib/ctdbd_conn.c
index 84bba3b..6b50009 100644
--- a/source3/lib/ctdbd_conn.c
+++ b/source3/lib/ctdbd_conn.c
@@ -361,10 +361,18 @@ static NTSTATUS ctdb_read_req(struct ctdbd_connection *conn, uint32 reqid,
goto next_pkt;
}
- if (msg->srvid == CTDB_SRVID_RECONFIGURE) {
- DEBUG(0,("Got cluster reconfigure message in ctdb_read_req\n"));
+ if ((msg->srvid == CTDB_SRVID_RECONFIGURE)
+ || (msg->srvid == CTDB_SRVID_SAMBA_NOTIFY)) {
+
+ DEBUG(1, ("ctdb_read_req: Got %s message\n",
+ (msg->srvid == CTDB_SRVID_RECONFIGURE)
+ ? "cluster reconfigure" : "SAMBA_NOTIFY"));
+
messaging_send(conn->msg_ctx, procid_self(),
MSG_SMB_BRL_VALIDATE, &data_blob_null);
+ messaging_send(conn->msg_ctx, procid_self(),
+ MSG_DBWRAP_G_LOCK_RETRY,
+ &data_blob_null);
TALLOC_FREE(hdr);
goto next_pkt;
}
@@ -493,6 +501,11 @@ NTSTATUS ctdbd_messaging_connection(TALLOC_CTX *mem_ctx,
goto fail;
}
+ status = register_with_ctdbd(conn, CTDB_SRVID_SAMBA_NOTIFY);
+ if (!NT_STATUS_IS_OK(status)) {
+ goto fail;
+ }
+
*pconn = conn;
return NT_STATUS_OK;
@@ -501,6 +514,16 @@ NTSTATUS ctdbd_messaging_connection(TALLOC_CTX *mem_ctx,
return status;
}
+struct messaging_context *ctdb_conn_msg_ctx(struct ctdbd_connection *conn)
+{
+ return conn->msg_ctx;
+}
+
+int ctdbd_conn_get_fd(struct ctdbd_connection *conn)
+{
+ return packet_get_fd(conn->pkt);
+}
+
/*
* Packet handler to receive and handle a ctdb message
*/
@@ -533,24 +556,22 @@ static NTSTATUS ctdb_handle_message(uint8_t *buf, size_t length,
SMB_ASSERT(conn->msg_ctx != NULL);
- if (msg->srvid == CTDB_SRVID_RECONFIGURE) {
+ if ((msg->srvid == CTDB_SRVID_RECONFIGURE)
+ || (msg->srvid == CTDB_SRVID_SAMBA_NOTIFY)){
DEBUG(0,("Got cluster reconfigure message\n"));
/*
- * when the cluster is reconfigured, we need to clean the brl
- * database
+ * when the cluster is reconfigured or someone of the
+ * family has passed away (SAMBA_NOTIFY), we need to
+ * clean the brl database
*/
messaging_send(conn->msg_ctx, procid_self(),
MSG_SMB_BRL_VALIDATE, &data_blob_null);
- /*
- * it's possible that we have just rejoined the cluster after
- * an outage. In that case our pending locks could have been
- * removed from the lockdb, so retry them once more
- */
- message_send_all(conn->msg_ctx, MSG_SMB_UNLOCK, NULL, 0, NULL);
+ messaging_send(conn->msg_ctx, procid_self(),
+ MSG_DBWRAP_G_LOCK_RETRY,
+ &data_blob_null);
TALLOC_FREE(buf);
-
return NT_STATUS_OK;
}
@@ -1310,6 +1331,50 @@ NTSTATUS ctdbd_control_local(struct ctdbd_connection *conn, uint32 opcode,
return ctdbd_control(conn, CTDB_CURRENT_NODE, opcode, srvid, flags, data, mem_ctx, outdata, cstatus);
}
+NTSTATUS ctdb_watch_us(struct ctdbd_connection *conn)
+{
+ struct ctdb_client_notify_register reg_data;
+ size_t struct_len;
+ NTSTATUS status;
+ int cstatus;
+
+ reg_data.srvid = CTDB_SRVID_SAMBA_NOTIFY;
+ reg_data.len = 1;
+ reg_data.notify_data[0] = 0;
+
+ struct_len = offsetof(struct ctdb_client_notify_register,
+ notify_data) + reg_data.len;
+
+ status = ctdbd_control_local(
+ conn, CTDB_CONTROL_REGISTER_NOTIFY, conn->rand_srvid, 0,
+ make_tdb_data((uint8_t *)®_data, struct_len),
+ NULL, NULL, &cstatus);
+ if (!NT_STATUS_IS_OK(status)) {
+ DEBUG(1, ("ctdbd_control_local failed: %s\n",
+ nt_errstr(status)));
+ }
+ return status;
+}
+
+NTSTATUS ctdb_unwatch(struct ctdbd_connection *conn)
+{
+ struct ctdb_client_notify_deregister dereg_data;
+ NTSTATUS status;
+ int cstatus;
+
+ dereg_data.srvid = CTDB_SRVID_SAMBA_NOTIFY;
+
+ status = ctdbd_control_local(
+ conn, CTDB_CONTROL_DEREGISTER_NOTIFY, conn->rand_srvid, 0,
+ make_tdb_data((uint8_t *)&dereg_data, sizeof(dereg_data)),
+ NULL, NULL, &cstatus);
+ if (!NT_STATUS_IS_OK(status)) {
+ DEBUG(1, ("ctdbd_control_local failed: %s\n",
+ nt_errstr(status)));
+ }
+ return status;
+}
+
#else
NTSTATUS ctdbd_init_connection(TALLOC_CTX *mem_ctx,
diff --git a/source3/lib/dbwrap_ctdb.c b/source3/lib/dbwrap_ctdb.c
index 8e188d0..938a312 100644
--- a/source3/lib/dbwrap_ctdb.c
+++ b/source3/lib/dbwrap_ctdb.c
@@ -1,7 +1,8 @@
/*
Unix SMB/CIFS implementation.
Database interface wrapper around ctdbd
- Copyright (C) Volker Lendecke 2007
+ Copyright (C) Volker Lendecke 2007-2009
+ Copyright (C) Michael Adam 2009
This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
@@ -22,10 +23,10 @@
#include "ctdb.h"
#include "ctdb_private.h"
#include "ctdbd_conn.h"
+#include "g_lock.h"
struct db_ctdb_transaction_handle {
struct db_ctdb_ctx *ctx;
- bool in_replay;
/*
* we store the reads and writes done under a transaction:
* - one list stores both reads and writes (m_all),
@@ -35,6 +36,7 @@ struct db_ctdb_transaction_handle {
struct ctdb_marshall_buffer *m_write;
uint32_t nesting;
bool nested_cancel;
+ char *lock_name;
};
struct db_ctdb_ctx {
@@ -42,6 +44,7 @@ struct db_ctdb_ctx {
struct tdb_wrap *wtdb;
uint32 db_id;
struct db_ctdb_transaction_handle *transaction;
+ struct g_lock_ctx *lock_ctx;
};
struct db_ctdb_rec {
@@ -49,11 +52,6 @@ struct db_ctdb_rec {
struct ctdb_ltdb_header header;
};
-static struct db_record *fetch_locked_internal(struct db_ctdb_ctx *ctx,
- TALLOC_CTX *mem_ctx,
- TDB_DATA key,
- bool persistent);
-
static NTSTATUS tdb_error_to_ntstatus(struct tdb_context *tdb)
{
NTSTATUS status;
@@ -297,145 +295,21 @@ static struct ctdb_rec_data *db_ctdb_marshall_loop_next(struct ctdb_marshall_buf
return r;
}
-
-static int32_t db_ctdb_transaction_active(uint32_t db_id)
-{
- int32_t status;
- NTSTATUS ret;
- TDB_DATA indata;
-
- indata.dptr = (uint8_t *)&db_id;
- indata.dsize = sizeof(db_id);
-
- ret = ctdbd_control_local(messaging_ctdbd_connection(),
- CTDB_CONTROL_TRANS2_ACTIVE, 0, 0,
- indata, NULL, NULL, &status);
-
- if (!NT_STATUS_IS_OK(ret)) {
- DEBUG(2, ("ctdb control TRANS2_ACTIVE failed\n"));
- return -1;
- }
-
- return status;
-}
-
-
/**
* CTDB transaction destructor
*/
static int db_ctdb_transaction_destructor(struct db_ctdb_transaction_handle *h)
{
- tdb_transaction_cancel(h->ctx->wtdb->tdb);
- return 0;
-}
-
-/**
- * start a transaction on a ctdb database:
- * - lock the transaction lock key
- * - start the tdb transaction
- */
-static int db_ctdb_transaction_fetch_start(struct db_ctdb_transaction_handle *h)
-{
- struct db_record *rh;
- struct db_ctdb_rec *crec;
- TDB_DATA key;
- TALLOC_CTX *tmp_ctx;
- const char *keyname = CTDB_TRANSACTION_LOCK_KEY;
- int ret;
- struct db_ctdb_ctx *ctx = h->ctx;
- TDB_DATA data;
- pid_t pid;
NTSTATUS status;
- struct ctdb_ltdb_header header;
- int32_t transaction_status;
-
- key.dptr = (uint8_t *)discard_const(keyname);
- key.dsize = strlen(keyname);
-again:
- tmp_ctx = talloc_new(h);
-
- rh = fetch_locked_internal(ctx, tmp_ctx, key, true);
- if (rh == NULL) {
- DEBUG(0,(__location__ " Failed to fetch_lock database\n"));
- talloc_free(tmp_ctx);
- return -1;
- }
- crec = talloc_get_type_abort(rh->private_data, struct db_ctdb_rec);
-
- transaction_status = db_ctdb_transaction_active(ctx->db_id);
- if (transaction_status == 1) {
- unsigned long int usec = (1000 + random()) % 100000;
- DEBUG(3, ("Transaction already active on db_id[0x%08x]."
- "Re-trying after %lu microseconds...",
- ctx->db_id, usec));
- talloc_free(tmp_ctx);
- usleep(usec);
- goto again;
- }
-
- /*
- * store the pid in the database:
- * it is not enought that the node is dmaster...
- */
- pid = getpid();
- data.dptr = (unsigned char *)&pid;
- data.dsize = sizeof(pid_t);
- crec->header.rsn++;
- crec->header.dmaster = get_my_vnn();
- status = db_ctdb_ltdb_store(ctx, key, &(crec->header), data);
+ status = g_lock_unlock(h->ctx->lock_ctx, h->lock_name);
if (!NT_STATUS_IS_OK(status)) {
- DEBUG(0, (__location__ " Failed to store pid in transaction "
- "record: %s\n", nt_errstr(status)));
- talloc_free(tmp_ctx);
- return -1;
- }
-
- talloc_free(rh);
-
- ret = tdb_transaction_start(ctx->wtdb->tdb);
- if (ret != 0) {
- DEBUG(0,(__location__ " Failed to start tdb transaction\n"));
- talloc_free(tmp_ctx);
+ DEBUG(0, ("g_lock_unlock failed: %s\n", nt_errstr(status)));
return -1;
}
-
- status = db_ctdb_ltdb_fetch(ctx, key, &header, tmp_ctx, &data);
- if (!NT_STATUS_IS_OK(status)) {
- DEBUG(0, (__location__ " failed to refetch transaction lock "
- "record inside transaction: %s - retrying\n",
- nt_errstr(status)));
- tdb_transaction_cancel(ctx->wtdb->tdb);
- talloc_free(tmp_ctx);
- goto again;
- }
-
- if (header.dmaster != get_my_vnn()) {
- DEBUG(3, (__location__ " refetch transaction lock record : "
- "we are not dmaster any more "
- "(dmaster[%u] != my_vnn[%u]) - retrying\n",
- header.dmaster, get_my_vnn()));
- tdb_transaction_cancel(ctx->wtdb->tdb);
- talloc_free(tmp_ctx);
- goto again;
--
Samba Shared Repository
More information about the samba-cvs
mailing list