inotify on Samba4 and clustering
jpeach at samba.org
Wed May 31 02:14:41 GMT 2006
On Tue, 2006-05-30 at 16:38 -0500, Abhijith Das wrote:
> Hi list,
> I was looking at the latest Samba4 code and found some use of the
> inotify interface. I was told before that dnotify was being replaced by
> inotify, so this is not a surprise.
> However, when I think of a clusterized implementation of inotify for
> GFS, I see complications.
> - An event-watch on one cluster node should be propagated to all nodes.
> - When an event occurs on a different node, we need to somehow write to
> the event-queue of the first node. This could mean some serious hacking.
> - There are way too many events to use a distributed-lock-manager's
> locking for internode communication. A separate userland event/messaging
> framework would have to deal with these event notifications and pass
> them around as necessary.
If you have a cluster architecture where a single node arbitrates
metadata access, you can have a system where the notification is applied
to the arbiter and the arbiter triggers the notification on behalf of
clients. I think there will might be some holes in this scheme since
clients can do I/O without notifying the arbiter immediately. It should
be a 90% solution, though.
> Is the effort worth the reward? In what situations does samba use
> inotify and what could be the performance impact without inotify support?
inotify is there to support the FindFirstFileChangeNotification call. As
Andrew says, this is a correctness issue, and there may be apps that
depend on this call working correctly. In a cluster it might be OK to
skip it if there is a known set of qualified apps running.
> An alternate solution could be a local-inotify implementation. We could
> add support in clustered-samba to pass these inotify messages to various
> cluster nodes as appropriate?
This might be a reasonable option. You already need something similar
for clusters that don't support oplocks. You'd need to register a watch
with the locking leader who would then force all the nodes to apply a
local watch .. or something like that.
James Peach | jpeach at samba.org
More information about the samba-technical