[cifs-protocol] [REG:115070612914524] What is FILE_ACTION_REMOVED_BY_DELETE? MS-FSCC 2.4.42

Edgar Olougouna edgaro at microsoft.com
Mon Oct 26 16:17:17 UTC 2015 

Volker,

Just an FYI, this is the provisional change to MS-FSCC and MS-FSA. The change will appear in future releases of the documents.

MS-FSCC

In section 2.4.42	FileNotifyInformation the description of FILE_ACTION_REMOVED_BY_DELETE has been changed from:

The file was removed by delete.

TO

An object ID was removed because the file the object ID referred to was deleted.

This notification is only sent when the directory being monitored is the special directory "\$Extend\$ObjId:$O:$INDEX_ALLOCATION".<125>

<125> Section 2.4.42:  Only NTFS supports this special directory.

MS-FSA

In section 2.1.1	Abstract Data Model two new ADM elements were added:

ViewIndexFile: A File object with a FileType of ViewIndexFile.
ViewIndexStream: A Stream object with a StreamType of ViewIndexStream.

In section 2.1.1.1	Per Volume a new ADM element was added:

§	IsPhysicalRoot: A Boolean that is TRUE if RootDirectory represents the root of the physical media format.

In section 2.1.1.3	Per File the ADM element FileType was changed from:

§	FileType: The type of file. This value MUST be either DataFile or DirectoryFile.

To

§	FileType: The type of file. This value MUST be DataFile, DirectoryFile, or ViewIndexFile.<11>
<11> Section 2.1.1.3:  Only NTFS supports view index files.

In section 2.1.1.5	Per Stream the ADM element StreamType was changed from:

§	StreamType: The type of stream. This value MUST be either DataStream or DirectoryStream.

To

§	StreamType: The type of stream. This value MUST be DataStream, DirectoryStream, or ViewIndexStream. <27>
<27> Section 2.1.1.5:  Only NTFS supports view index streams.

In section 2.1.1.8	Per ChangeNotifyEntry the ADM element	OpenedDirectory has been changed from:

§	OpenedDirectory: The Open of the DirectoryFile to monitor for changes.

To

§	OpenedDirectory: The Open of the DirectoryFile or ViewIndexFile to monitor for changes.

In section 2.1.5.1	Server Requests an Open of a File the bullets under Phase 5 have been changed from:

§	The object store MUST split Open.FileName into pathname components PathName1 ... PathNamen, using the backslash ("\") character as a delimiter. The object store MUST further split each PathNamei into a file name component FileNamei, stream name component StreamNamei, and stream type name component StreamTypeNamei, using the colon (":") character as a delimiter (FileNamei:StreamNamei:StreamTypeNamei). If StreamNamei or StreamTypeNamei is not present in the name, the value MUST be set to an empty string.
§	If any StreamTypeNamei is "$INDEX_ALLOCATION" and the corresponding StreamNamei has a value other than an empty string or "$I30", the operation SHOULD be failed with STATUS_INVALID_PARAMETER.

To

§	The object store MUST split Open.FileName into pathname components PathName1 ... PathNamen, using the backslash ("\") character as a delimiter. If any PathNamei ends in a colon(":") character, the operation MUST be failed with STATUS_OBJECT_NAME_INVALID. The object store MUST further split each PathNamei into a file name component FileNamei, stream name component StreamNamei, and stream type name component StreamTypeNamei, using the colon (":") character as a delimiter (FileNamei:StreamNamei:StreamTypeNamei). If StreamNamei or StreamTypeNamei is not present in the name, the value MUST be set to an empty string.

The for loop in phase 6 has been changed from:

§	For i = 1 to n-1: // n is the number of pathname components, from Phase 5.
§	Search ParentFile.DirectoryList for a Link where Link.Name or Link.ShortName matches FileNamei, If no such link is found, the operation MUST be failed with STATUS_OBJECT_PATH_NOT_FOUND.
§	If Link.File.FileType is not DirectoryFile, the operation MUST be failed with STATUS_NOT_A_DIRECTORY.
§	If Open.GrantedAccess.FILE_TRAVERSE is not set and AccessCheck(SecurityContext, Link.File.SecurityDescriptor, FILE_TRAVERSE) returns FALSE, the operation MAY be failed with STATUS_ACCESS_DENIED.
§	If Link.IsDeleted, the operation MUST be failed with STATUS_DELETE_PENDING.
§	If Link.File.IsSymbolicLink is TRUE, the operation MUST be failed with Status set to STATUS_STOPPED_ON_SYMLINK and ReparsePointData set to Link.File.ReparsePointData.
§	Set ParentFile = Link.File.
§	EndFor

To

§	For i = 1 to n-1: // n is the number of pathname components, from Phase 5.
§	If StreamTypeNamei is non-empty:
§	Set ComplexNameSuffix = ":StreamNamei:StreamTypeNameii".
§	Else if StreamTypeNamei is non-empty:
§	Set ComplexNameSuffix = ":StreamNamei".
§	Else:
§	Set ComplexNameSuffix to empty.
§	EndIf
§	If ComplexNameSuffix is non-empty and ComplexNameSuffix is not equal to one of the complex name suffixes recognized by the object store<35> (using case-insensitive string comparisons), the operation MUST be failed with STATUS_OBJECT_NAME_INVALID.
§	Search ParentFile.DirectoryList for a Link where Link.Name or Link.ShortName matches FileNamei. If no such link is found, the operation MUST be failed with STATUS_OBJECT_PATH_NOT_FOUND.
§	If Link.File.FileType is not DirectoryFile, the operation MUST be failed with STATUS_NOT_A_DIRECTORY.
§	If Open.GrantedAccess.FILE_TRAVERSE is not set and AccessCheck(SecurityContext, Link.File.SecurityDescriptor, FILE_TRAVERSE) returns FALSE, the operation MAY be failed with STATUS_ACCESS_DENIED.
§	If Link.IsDeleted, the operation MUST be failed with STATUS_DELETE_PENDING.
§	If Link.File.IsSymbolicLink is TRUE, the operation MUST be failed with Status set to STATUS_STOPPED_ON_SYMLINK and ReparsePointData set to Link.File.ReparsePointData.
§	Set ParentFile = Link.File.
§	EndFor

<35> Section 2.1.5.1:  NTFS and ReFS recognize the following complex name suffixes:
§	":$I30"
§	"::$INDEX_ALLOCATION"
§	":$I30:$INDEX_ALLOCATION"
§	"::$BITMAP"
§	":$I30:$BITMAP"
§	"::$ATTRIBUTE_LIST"
§	"::$REPARSE_POINT"
Other Windows file systems do not recognize any complex name suffixes.

Phase 7 has been changed from:

§	Phase 7 -- Type of file to open:
§	The object store MUST use the following algorithm to determine which type of file is being opened:
§	If CreateOptions.FILE_DIRECTORY_FILE is TRUE then FileTypeToOpen = DirectoryFile.
§	Else if CreateOptions.FILE_NON_DIRECTORY_FILE is TRUE then FileTypeToOpen = DataFile.
§	Else if StreamTypeNameToOpen is "$INDEX_ALLOCATION" then FileTypeToOpen = DirectoryFile.
§	Else if StreamTypeNameToOpen is "$DATA" then FileTypeToOpen = DataFile.
§	Else if Open.File is not NULL and Open.File.FileType is DirectoryFile, then FileTypeToOpen = DirectoryFile.
§	Else if PathName contains a trailing backslash then FileTypeToOpen = DirectoryFile.
§	Else FileTypeToOpen = DataFile.
§	If FileTypeToOpen is DirectoryFile and Open.File is not NULL and Open.File.FileType is not DirectoryFile:
§	If CreateDisposition == FILE_CREATE then the operation MUST be failed with STATUS_OBJECT_NAME_COLLISION, else the operation MUST be failed with STATUS_NOT_A_DIRECTORY.
§	EndIf
§	If FileTypeToOpen is DataFile and StreamNameToOpen is empty and Open.File is not NULL and Open.File.FileType is DirectoryFile, the operation MUST be failed with STATUS_FILE_IS_A_DIRECTORY.

To

§	Phase 7 -- Type of file to open:
§	The object store MUST use the following algorithm to determine which type of file is being opened:
§	Set FileTypeToOpen to empty.
§	If RootOpen.File.Volume.IsPhysicalRoot is TRUE, then set FileTypeToOpen to ViewIndexFile under any of the following conditions:
§	If RootOpen.File.Volume.IsObjectIDsSupported is TRUE, BuildRelativeName(Open.Link, Open.File.Volume.RootDirectory) is equal to "\$Extend\$ObjId", StreamNameToOpen is equal to "$O", and StreamTypeNameToOpen is equal to "$INDEX_ALLOCATION" (using case-insensitive string comparisons).
§	If RootOpen.File.Volume.IsQuotasSupported is TRUE, BuildRelativeName(Open.Link, Open.File.Volume.RootDirectory) is equal to "\$Extend\$Quota", StreamNameToOpen is equal to "$O" or "$Q", and StreamTypeNameToOpen is equal to "$INDEX_ALLOCATION" (using case-insensitive string comparisons).
§	If RootOpen.File.Volume.IsReparsePointsSupported is TRUE, BuildRelativeName(Open.Link, Open.File.Volume.RootDirectory) is equal to "\$Extend\$Reparse", StreamNameToOpen is equal to "$R", and StreamTypeNameToOpen is equal to "$INDEX_ALLOCATION" (using case-insensitive string comparisons).
§	EndIf
§	// Note that when FileTypeToOpen is ViewIndexFile, the file always exists in the object store and Open.File.FileType is ViewIndexFile.
§	If FileTypeToOpen is empty:
§	If StreamTypeNameToOpen is "$INDEX_ALLOCATION" and StreamNameToOpen has a value other than an empty stream or "$I30", the operation MUST be failed with STATUS_INVALID_PARAMETER.
§	If CreateOptions.FILE_DIRECTORY_FILE is TRUE then FileTypeToOpen = DirectoryFile.
§	Else if CreateOptions.FILE_NON_DIRECTORY_FILE is TRUE then FileTypeToOpen = DataFile.
§	Else if StreamTypeNameToOpen is "$INDEX_ALLOCATION" then FileTypeToOpen = DirectoryFile.
§	Else if StreamTypeNameToOpen is "$DATA" then FileTypeToOpen = DataFile.
§	Else if Open.File is not NULL and Open.File.FileType is DirectoryFile, then FileTypeToOpen = DirectoryFile.
§	Else if PathName contains a trailing backslash then FileTypeToOpen = DirectoryFile.
§	Else FileTypeToOpen = DataFile.
§	EndIf
§	If FileTypeToOpen is DirectoryFile and Open.File is not NULL and Open.File.FileType is not DirectoryFile:
§	If CreateDisposition == FILE_CREATE then the operation MUST be failed with STATUS_OBJECT_NAME_COLLISION, else the operation MUST be failed with STATUS_NOT_A_DIRECTORY.
§	EndIf
§	If FileTypeToOpen is DataFile and StreamNameToOpen is empty and Open.File is not NULL and Open.File.FileType is DirectoryFile, the operation MUST be failed with STATUS_FILE_IS_A_DIRECTORY.

In section 2.1.5.1.2	Open of an Existing File the following has been added

§	Else:  // FileTypeToOpen is ViewIndexFile
§	// Note that when FileTypeToOpen is ViewIndexFile, the stream always exists in the file and Open.Stream.StreamType is ViewIndexStream.

In section 2.1.5.4	Server Requests Closing an Open the following has been added at the end of Phase 5

§	If FileDeleted is TRUE and Open.File.ObjectId is not empty:
§	Send directory change notification as per section 2.1.4.1, with Volume equal to Open.File.Volume, Action equal to FILE_ACTION_REMOVED_BY_DELETE, FilterMatch equal to FILE_NOTIFY_CHANGE_FILE_NAME, and FileName equal to "\$Extend\$ObjId".
§	EndIf

In section 2.1.5.9.1	FSCTL_CREATE_OR_GET_OBJECT_ID the following has been added to the end of the if statement

§	Send directory change notification as per section 2.1.4.1, with Volume equal to Open.File.Volume, Action equal to FILE_ACTION_ADDED, FilterMatch equal to FILE_NOTIFY_CHANGE_FILE_NAME, and FileName equal to "\$Extend\$ObjId".

In section 2.1.5.9.2	FSCTL_DELETE_OBJECT_ID the following has been added

§	Send directory change notification as per section 2.1.4.1, with Volume equal to Open.File.Volume, Action equal to FILE_ACTION_REMOVED, FilterMatch equal to FILE_NOTIFY_CHANGE_FILE_NAME, and FileName equal to "\$Extend\$ObjId".

In section 2.1.5.9.31	FSCTL_SET_OBJECT_ID the following has been added

§	Send directory change notification as per section 2.1.4.1, with Volume equal to Open.File.Volume, Action equal to FILE_ACTION_ADDED, FilterMatch equal to FILE_NOTIFY_CHANGE_FILE_NAME, and FileName equal to "\$Extend\$ObjId".

In section 2.1.5.10	Server Requests Change Notifications for a Directory Open has been changed from:

§	Open: An Open of a DirectoryStream.

To

§	Open: An Open of a DirectoryStream or ViewIndexStream.

Thanks,
Edgar

-----Original Message-----
From: Edgar Olougouna 
Sent: Thursday, July 23, 2015 1:58 AM
To: Volker.Lendecke at SerNet.DE
Cc: Richard Sharpe <realrichardsharpe at gmail.com>; cifs-protocol at lists.samba.org; MSSolve Case Email <casemail at microsoft.com>
Subject: RE: [REG:115070612914524] [cifs-protocol] What is FILE_ACTION_REMOVED_BY_DELETE? MS-FSCC 2.4.42

Volker,
FILE_ACTION_REMOVED_BY_DELETE does not apply to ReFS, which, as you mentioned, does not support object IDs.

Thanks,
Edgar

-----Original Message-----
From: Volker Lendecke [mailto:Volker.Lendecke at SerNet.DE] 
Sent: Monday, July 20, 2015 12:05 AM
To: Edgar Olougouna <edgaro at microsoft.com>
Cc: Richard Sharpe <realrichardsharpe at gmail.com>; cifs-protocol at lists.samba.org; MSSolve Case Email <casemail at microsoft.com>
Subject: Re: [REG:115070612914524] [cifs-protocol] What is FILE_ACTION_REMOVED_BY_DELETE? MS-FSCC 2.4.42

On Sat, Jul 18, 2015 at 08:10:46PM +0000, Edgar Olougouna wrote:
> Volker,
> 
> Per the NTFS source code and confirmation with file system team, the FILE_ACTION_REMOVED_BY_DELETE is posted for a notification whenever we delete the object ID attribute from a file and remove that object ID from the index. In this case we do not post a change to the USN journal.  
> 
> FILE_ACTION_REMOVED is used for other cases, as documented in MS-FSA (such as when a file is deleted).
> 
> Let's me know whether this answers your question. I have logged a technical document bug on MS-FSA. 

Well, kindof yes.

This means that for example on ReFS this is never generated, because ReFS does not implement object IDs?

I'll probably have to wait for the next iteration of [MS-FSA], but I'm curious about the server actions that trigger this.

Volker

--
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9 AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen http://www.sernet.de, mailto:kontakt at sernet.de

More information about the cifs-protocol mailing list