Is the sky actually falling? (NTFS streams)

[Henrik Nordstrom]

On Sat, 24 Sep 2005, Andrew Bartlett wrote:

> - Upload the streamed file to a website?
> - Send the streamed file as a standard e-mail attachment?
> - Place the streamed file in a zip archive?
> - Burn the streamed file to a CD?
> - Backup with standard backup software?  (I expect at least this is
> possible)

In nearly all cases including many backup software you only get the 
default stream, not the named streams.

And it's not fair to blaim UNIX for this. The custom of seeing files as a 
single stream of data predates UNIX by far, and is also what the Windows 
world is most used to. The streams support in Windows is a fairly recent 
addition only available starting with NT3.1 and NTFS, and has only very 
recently found any real uses within the Windows world.

Some interesting reading on the subject from Microsoft:

http://msdn.microsoft.com/library/en-us/dnfiles/html/ntfs5.asp

> Assuming that streams are silently stripped when doing these things, I
> suggest that the author of any software that relies on streams will find
> their users silently frustrated.

Indeed, and is why streams is mostly useful for storing information 
related to the file but not actually part of the real content of the file. 
This is security attributes, annotations, file type, which application 
should be used to open the file, etc..

But it can be highly expected more and more applications starts to make 
use of streams for storing this kind of information. Information you would 
not get killed for if it was not preserved in a backup or copy but which 
still makes life a bit better.

> Will Windows users put up with the pain Mac users endured, where every 
> file had to be packed with stuff-it before leaving the Mac?

Not even the MAC users does really. But it is also the case that far from 
every file has to be packeted before leaving a Mac. Very many file formats 
correctly stores all document data in the main file (data fork) and only 
other "derived" data (file type, preview, icon, application configuration 
settings, clipboard contents etc) in the resource fork, and this is in 
fact how the Mac file system is supposed to be used. Then there is 
obviously a few exceptions where applications for some strange reason 
stores important document information in the resouce fork but from a file 
format design perspective these applications can be argued to be broken..

> Instead of being 'every office document', I instead expect this will be
> an issue at large, specialist sites where the IE security tagging or
> custom software become the showstopper.  With EAs becoming standard for
> SELinux and reiser4 pushing again, I think we can handle these
> specialist cases too.

Not so sure. There is a lot more data to a file than just the actual 
contents, and I do expect more and more applications to make use of the 
ability to store such information when possible as now even Microsoft has 
understood that using streams for storing associated information about the 
content in streams is quite useful and makes it a lot easier to deal with 
this kind of information in a general manner not dependent on file type.

But I think it will remain for a long time that these streams is seen 
mainly as a "bonus". But it is quite likely there will be applications 
nearly requiring streams support during day to day use (but hopefully not 
for transfer of files between users/systems).

Regards
Henrik