[Samba] 答复: Is it fine to store the tdb files in the shared directory to solve the ctdb data synchronization problem?

[Volker Lendecke]

On Fri, Dec 02, 2016 at 12:20:14PM +0000, Chenyehua via samba wrote:
> Thanks very much, Volker.
> 
> I want to synchronize three tdb files(listed as below) which is relevant to the opened file:
> 1) brlock.tdb
> 2) locking.tdb
> 3) smbXsrv_open_global.tdb
> These three tdb files are used to realize the durable file handle feature.

Ah, ok, you want to implement durable file handles in a cluster.

> As what you have said, the performance will be affected if storing
> these three files in the shared storage.
> So can I realize the synchronization with the solutions below:
> 1) set these three tdb files as persistent tdb files to realize
> synchronization (Is this way even worse than storing in the shared
> storage when concerning the performance affecting?)

Making them persistent will make it even worse than putting them into
shared storage I guess. For durable file handles in a shared storage
you need local fsyncs, because for normal tdb operations it is highly
unlikely that a tdb modification will immediately end up on a shared
disk. So it won't survive a node crash without a fsync call.

> 2) Real-time backing up the modification of the normal tdb to other
> ctdb nodes(just a simple guess, not test yet)

Yep, that's one way to do it. We have considered making those tdbs
semi-persistent: Use the same mechanisms that are used for persistent
databases but without the fsyncs. I haven't measured it, but I would
guess that the main cost of persistent database modification is the
local fsync that every node has to wait for.

> Do you have any other solutions or suggestions about this data
> synchronization problem?

Well, I've been working on a separate database model: I have some very
early (and not yet fully working) code that can make every database
modification an isolated transaction style modification. The main
point is that it won't use fcntl locks or mutexes. Once that is in, it
will be possible to delegate the database modifications to some sort
of consensus based clustered hash table. If we'll write that on our
own or make it a commodity nosql db, not sure yet. But the first thing
will be to get rid of the os-level hash chain locks, so that we can
turn the database into a log-based finite state machine that can more
easily replicated. Nothing for this or next week, but that's what I am
exploring when I find the time.

Hope that helps,

Volker