Patch to support Scalable CTDB

[Partha Sarathi]

Hi,

We have a requirement to support a large cluster i.e scaling from 20 nodes
to 50 nodes cluster and the current CTDB design may not support the linear
scaling of nodes in the cluster as the replication of all lock
related tdbs and tdb traversing for every record may slow down the
performance.

 The product requirement for creating large number nodes say 50 in a
cluster with subgrouping them into three/five nodes into
multiple protocol heads. Each of these protocol head group nodes will host
a specific set of shares not all.  So we took an approach to create two
instances of CTDBD on each node.

1) The primary ctdbd (Persistent ctdbd) is responsible to just replicate
the persistent TDBs across the cluster in our case 50 nodes to maintain the
AD registered account details and supporting single global namespace across
the large cluster.

2) The secondary instance called ( Locking ctdbd) is responsible
to replicate and traverse the lock related TDBs within the protocol heads
group in that way reducing the latency TDB transactions (expensive when the
number of nodes is large) by just communicating within the limited nodes
group.

3) Smbd changed in such a way that it communicates against these two
instance of CTDBDs with different ctdbd sockets. The message inits and the
connection handling have been well-taken care.

To have the above ctdbd running independently they are configured
separately and listening on different ctdb ports (4379 and 4380)
respectively.



With the attached patch, I could see and get the expected things running as
below

root at oneblox1:/opt/exablox/config# ctdb -s
/var/run/ctdb/persistent_ctdbd.socket getdbmap
Number of databases:7
dbid:0x88d96253 name:smbpasswd.tdb
path:/exablox/pctdb/persistent/smbpasswd.tdb.1 PERSISTENT
dbid:0x2ca251cf name:account_policy.tdb
path:/exablox/pctdb/persistent/account_policy.tdb.1 PERSISTENT
dbid:0xa1413774 name:group_mapping.tdb
path:/exablox/pctdb/persistent/group_mapping.tdb.1 PERSISTENT
dbid:0xc3078fba name:share_info.tdb
path:/exablox/pctdb/persistent/share_info.tdb.1 PERSISTENT
dbid:0x6cf2837d name:registry.tdb
path:/exablox/pctdb/persistent/registry.tdb.1 PERSISTENT
*dbid:0x7132c184 name:secrets.tdb
path:/exablox/pctdb/persistent/secrets.tdb.1 PERSISTENT*
*dbid:0x6645c6c4 name:ctdb.tdb path:/exablox/pctdb/persistent/ctdb.tdb.1
PERSISTENT*


root at oneblox1:/opt/exablox/config# ctdb -s /var/run/ctdb/ctdbd.socket
getdbmap
Number of databases:11
dbid:0x477d2e20 name:smbXsrv_client_global.tdb
path:/var/tmp/samba/ctdb/smbXsrv_client_global.tdb.1
dbid:0x66f71b8c name:smbXsrv_open_global.tdb
path:/var/tmp/samba/ctdb/smbXsrv_open_global.tdb.1
dbid:0x9ec2a880 name:serverid.tdb path:/var/tmp/samba/ctdb/serverid.tdb.1
dbid:0x06916e77 name:leases.tdb path:/var/tmp/samba/ctdb/leases.tdb.1
*dbid:0x7a19d84d name:locking.tdb path:/var/tmp/samba/ctdb/locking.tdb.1*
dbid:0x4e66c2b2 name:brlock.tdb path:/var/tmp/samba/ctdb/brlock.tdb.1
dbid:0x68c12c2c name:smbXsrv_tcon_global.tdb
path:/var/tmp/samba/ctdb/smbXsrv_tcon_global.tdb.1
dbid:0x6b06a26d name:smbXsrv_session_global.tdb
path:/var/tmp/samba/ctdb/smbXsrv_session_global.tdb.1
dbid:0x521b7544 name:smbXsrv_version_global.tdb
path:/var/tmp/samba/ctdb/smbXsrv_version_global.tdb.1
dbid:0x4d2a432b name:g_lock.tdb path:/var/tmp/samba/ctdb/g_lock.tdb.1
*dbid:0x6645c6c4 name:ctdb.tdb path:/exablox/ctdb/persistent/ctdb.tdb.1
PERSISTENT*


Please review the patch and let me know your comments.

Regards,
--Partha
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Support-two-instances-of-CTDBD-for-isolating-the-per.patch
Type: application/octet-stream
Size: 20328 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20180426/32f3482e/0001-Support-two-instances-of-CTDBD-for-isolating-the-per.obj>