Streams support in Linux

[Al Viro]

On Sat, Aug 25, 2018 at 08:51:50AM -0700, Matthew Wilcox wrote:

> > What do you do for access via /proc/*/fd/*, when descriptor is a sodding
> > stream?
> 
> I don't know.  Is it useful to be able to access them that way?

I don't know - I've no idea who needs that... little gem of elegance in the
first place.

> > > renameat()
> > > If olddirfd + oldpath refers to a stream then newdirfd + newpath must
> > > refer to a stream within the same parent object.  If that stream exists,
> > > it is removed.  If olddirfd + oldpath does not refer to a stream, then
> > > newdirfd + newpath must not refer to a stream.
> > 
> > And here the shit begins - what to do when parents have different dentries?
> > Directories can't have hardlinks; regular files very much can.
> 
> I presume you mean from a locking point of view?  In my mind, we check that
> old and new resolve to the same inode and take a lock on that inode.

Now, if that was all we ever needed...

> > Where do dentries of these things belong, anyway?
> 
> Do streams have to have dentries?  They don't have names in the hierarchy.
> Can they share their parent's dentry?  Can they have a NULL dentry?  Or
> a magic dentry of some kind?

If you want them as opened files - yes, they have to, no, they can't and no,
they can't.

> > > unlinkat()
> > > This is how you remove an individual named stream
> > 
> > Yeah... and what happens to access via other hardlinks to the same parent
> > file?
> 
> I'm not sure I understand the problem here.  You have, say /a/b/c
> and /d which refer to the same inode.  This inode has streams x and y.
> If you unlinkat() /a/b/c::x then it is no longer openable.  Any currently
> existing open fds for x keep x alive until the last one is closed, no
> matter which path was used to open them.

Again, where it the tree would the dentries live?  And I really hope you are
not suggesting to reserve '::' - not when we have files like
/usr/share/man/man3/Params::Check.3perl.gz
just to pull a random line out of locate :: output here...

> > > Unlinking a file with named streams removes all named streams from that
> > > file and then unlinks the file.  Open streams will continue to exist in
> > > the filesystem until they are closed, just as unlinked files do.
> > 
> > Can they be looked up by openat() from such a starting point?
> 
> No, you can't use openat() on a stream fd.  I should have been clearer
> on that above; this proposal should have split the syscall list in two;
> one for how the syscall behaves on a fd-for-a-file-with-streams and one
> for how the syscall behaves on a stream-fd.

I'm not suggesting passing a stream fd as the starting point; just a normal
opened-and-then-unlinked regular file.

> Right, but that code also assumes it's dealing with files.

I'm looking forward to your analysis of all code related to dentry tree,
with an eye towards the places where such asserts are quietly made...