[LFS/MM TOPIC] Enabling file and directory change notification for network and cluster file systems

[ronnie sahlberg]

On Tue, Jan 21, 2020 at 7:43 PM Amir Goldstein <amir73il at gmail.com> wrote:
>
> On Tue, Jan 21, 2020 at 10:30 AM ronnie sahlberg
> <ronniesahlberg at gmail.com> wrote:
> >
> > On Tue, Jan 21, 2020 at 5:48 PM Amir Goldstein <amir73il at gmail.com> wrote:
> > >
> > > On Tue, Jan 21, 2020 at 5:55 AM Steve French <smfrench at gmail.com> wrote:
> > > >
> > > > Currently the inotify interface in the kernel can only be used for
> > > > local file systems (unlike the previous change notify API used years
> > > > ago, and the change notify interface in Windows and other OS which is
> > > > primarily of interest for network file systems).
> > > >
> > > > I wanted to discuss the VFS changes needed to allow inotify requests
> > > > to be passed into file systems so network and cluster file systems (as
> > > > an example in the SMB3 case this simply means sending a
> > > > SMB3_CHANGE_NOTIFY request to the server, whether Samba or Cloud
> > > > (Azure) or Mac or Windows or Network Appliance - all support the API
> > > > on the server side, the problem is that the network or cluster fs
> > > > client isn't told about the request to wait on the inotify event).
> > > > Although user space tools can use file system specific ioctls to wait
> > > > on events, it is obviously preferable to allow network and cluster
> > > > file systems to wait on events using the calls which current Linux
> > > > GUIs use.
> > > >
> > > > This would allow gnome file manager GUI for example to be
> > > > automatically updated when a file is added to an open directory window
> > > > from another remote client.
> > > >
> > > > It would also fix the embarrassing problem noted in the inotify man page:
> > > >
> > > > "Inotify  reports  only events that a user-space program triggers
> > > > through the filesystem
> > > >        API.  As a result, it does not catch remote events that occur
> > > > on  network  filesystems."
> > > >
> > > > but that is precisely the types of notifications that are most useful
> > > > ... users often are aware of updates to local directories from the
> > > > same system, but ... automatic notifications that allow GUIs to be
> > > > updated on changes from **other** clients is of more value (and this
> > > > is exactly what the equivalent API allows on other OS).
> > > >
> > > > The changes to the Linux VFS are small.
> > > >
> > > >
> > >
> > > Miklos has already posted an RFC patch:
> > > https://lore.kernel.org/linux-fsdevel/20190507085707.GD30899@veci.piliscsaba.redhat.com/
> > >
> > > Did you try it?
> > >
> > > You also did not answer Miklos' question:
> > > does the smb protocol support whole filesystem (or subtree) notifications?
> > > (or just per-directory notifications)?
> >
> > SMB can do both. There is a flag that specifies if you want to just
> > get notified about the directory itself
> > or whether  you want notifications from the whole subtree.
> >
>
> I see. There is no user API in Linux to request a "subtree" watch.
> For the private case that the user requests a FAN_MARK_FILESYSTEM,
> cifs may translate that into a SMB2_WATCH_TREE for the share root dir.
>
> For that, Miklos' RFC of vfs interface inode->i_op->notify_update(inode)
> should be enriched with sb->s_op->notify_update(sb).

It all depends on what actual linux applications want/need.

On windows, things like the windows explorer file manager only uses
"this directory only" watches.
I imagine that linux gui filemanagers would do the same and be fine
with this directory only watches.

On windows you also have a large class of applications that does file
caching in the userspace application itself. Like IIS or other
applications
such as indexing services or things that keep track of changes to know
what to backup at a later stage.
They always use "watch the whole subtree" version of notifications.
I don't think there are any linux native applications that would
depend on this as the functionality was never present. Maybe an
exception would be windows applications running under wine, but surely
no currently existing linux native applications.

If we do get an API that would allow to watch an entire subtree then
that would make it possible for future backup applications to become
much more effective for incremental backups.  Things like "git status"
would likely also benefit greatly from such an api if it became
available.

>
> Thanks,
> Amir.