[cifs-protocol] [REG:112120310051084] FSCTL_SRV_COPYCHUNK overlapping ranges

Edgar Olougouna edgaro at microsoft.com
Thu Dec 20 15:20:12 MST 2012


David,
We have completed our investigation on this issue regarding server handling of SMB2 FSCTL_SRV_COPYCHUNK overlapping ranges. As a result a future release of MS-SMB2 will include the following updates.
The attached pdf to this email contains the changes with highlights of the updates.

Provisional MS-SMB2 changes
3.3.1.10   Per Open
Open.ResumeKey: A 24-byte key that identifies a source file in Server side Data Copy Operation.
3.3.5.15.5   Handling a Source File Key Request
...
The server MUST provide a 24-byte value that is used to uniquely identify the open. The server SHOULD use Open.DurableFileId, or alternately, MAY use an internally generated value that is unique for all opens on the server.<283>The server MUST set Open.ResumeKey and ResumeKey in SRV_REQUEST_RESUME_KEY Response to the generated value.

3.3.5.15.6   Handling a Server-Side Data Copy Request
When the server receives a request with an SMB2 header with a Command value equal to SMB2 IOCTL, and a CtlCode of FSCTL_SRV_COPYCHUNK or FSCTL_SRV_COPYCHUNK_WRITE, message handling proceeds as follows:
The server MUST locate the source open from where data will be read by locating the open where Open.ResumeKey matches SourceKey received in the SRV_COPYCHUNK_COPY structure received in the buffer described by InputCount and InputOffset of the SMB2 IOCTL Request. If the open is not found, the server MUST fail the request with STATUS_OBJECT_NAME_NOT_FOUND.
If OutputCount in SMB2 IOCTL request is less than the size of SRV_COPYCHUNK_RESPONSE structure, the server MUST fail the SMB2 IOCTL request with STATUS_INVALID_PARAMETER.
If OutputCount in SMB2 IOCTL request is greater or equal to the size of SRV_COPYCHUNK_RESPONSE structure and under any of the following conditions, the server MUST send an SMB2 IOCTL Response as specified in section 3.3.5.15.6.2:
*         InputCount in SMB2 IOCTL request is less than the size of Buffer field containing SRV_COPYCHUNK_COPY structure
*         ChunkCount is greater than ServerSideCopyMaxNumberofChunks
*         Length in a single chunk is greater than ServerSideCopyMaxChunkSize or equal to zero
*         Sum of Lengths in all chunks is greater than ServerSideCopyMaxDataSize
*         TargetOffset in any chunk is less than zero but not equal to 0xFFFFFFFFFFFFFFFF.
*         Open.TreeConnect of the source or destination file is on a named pipe filesystem
If Open.GrantedAccess of the destination file does not include FILE_WRITE_DATA or FILE_APPEND_DATA, then the request MUST be failed with STATUS_ACCESS_DENIED. If Open.GrantedAccess of the source file does not include FILE_READ_DATA access, then the request MUST be failed with STATUS_ACCESS_DENIED.
If Open.TreeConnect.Session of the destination file is not equal to Open.TreeConnect.Session of the source file, the server MUST fail the request with STATUS_OBJECT_NAME_NOT_FOUND.
Starting with the first chunk received in the Chunks field and for each chunk, the server MUST apply the following processing rules:
*        The server MUST issue a read using the SourceOffset and Length from the source file.<284> If the SourceOffset or SourceOffset + Length extends beyond the end of file, the server SHOULD<285> treat this as a STATUS_END_OF_FILE error. If the read fails, the server MUST map the error code returned to a valid status code as described in section 2.2, and MUST send an SMB2 IOCTL response as specified in Sending a Copy Failure Server-Side Copy Response (section 3.3.5.15.6.1).
*         If the read operation is successful, the server MUST issue a write of the data read using the TargetOffset and Length in the range against the destination file.<286> If the write fails, the server MUST send an SMB2 IOCTL response as specified in Sending a Copy Failure Server-Side Copy Response (section 3.3.5.15.6.1).
If all ranges are copied successfully, the server MUST construct an SMB2 IOCTL Response following the syntax specified in the section 2.2.32, with the following values:
* CtlCode MUST be set to FSCTL_SRV_COPYCHUNK or FSCTL_SRV_COPYCHUNK_WRITE.
* FileId.Persistent MUST be set to Open.DurableFileId. FileId.Volatile MUST be set to Open.FileId.
* InputOffset SHOULD be set to the offset, in bytes, from the beginning of the SMB2 header to the Buffer[] field of the response.
* InputCount SHOULD be set to zero.
* OutputOffset MUST be set to InputOffset + InputCount, rounded up to a multiple of 8.
* OutputCount MUST be set to 12.
* Flags MUST be set to zero.
* The server MUST copy a SRV_COPYCHUNK_RESPONSE following the syntax specified in section 2.2.32.1 into the Buffer field at the OutputOffset computed above. ChunksWritten MUST be set to the number of chunks processed. ChunkBytesWritten MUST be set to zero. TotalBytesWritten MUST be set to the total number of bytes written to the destination file across all chunk writes.
The response MUST be sent to the client.
3.3.5.15.6.1   Sending a Copy Failure Server-Side Copy Response
If a range is encountered that is not copied successfully, the server MUST construct an SMB2 IOCTL Response following the syntax specified in section 2.2.32, with the following values:
* Status in the SMB2 header MUST be set to the error returned during processing specified in section 3.3.5.15.6.
* CtlCode MUST be set to the CtlCode in SMB2 IOCTL Request.
* FileId.Persistent MUST be set to Open.DurableFileId. FileId.Volatile MUST be set to Open.FileId.
* InputOffset SHOULD be set to the offset, in bytes, from the beginning of the SMB2 header to the Buffer[] field of the response.
* InputCount SHOULD be set to zero.
* OutputOffset MUST be set to InputOffset + InputCount, rounded up to a multiple of 8.
* OutputCount MUST be set to 12.
* Flags MUST be set to zero.
* The server MUST copy a SRV_COPYCHUNK_RESPONSE following the syntax specified in section 2.2.32.1 into the Buffer field at the OutputOffset computed above. ChunksWritten MUST be set to the number of chunks successfully written. If the error was encountered partway through a write, ChunkBytesWritten MUST be set to the number of bytes written in the final, partial write. Otherwise, ChunkBytesWritten MUST be set to 0. TotalBytesWritten MUST be set to the total number of bytes written to the destination file across all chunk writes.
The response MUST be sent to the client.
3.3.5.15.6.2   Sending an Invalid Parameter Server-Side Copy Response
The server MUST construct an SMB2 IOCTL Response, following the syntax specified in section 2.2.32, with the following values:
* Status in the SMB2 header MUST be set to STATUS_INVALID_PARAMETER.
* CtlCode MUST be set to the CtlCode in SMB2 IOCTL Request.
* FileId.Persistent MUST be set to Open.DurableFileId. FileId.Volatile MUST be set to Open.FileId.
* InputOffset SHOULD be set to the offset, in bytes, from the beginning of the SMB2 header to the Buffer[] field of the response.
* InputCount SHOULD be set to zero.
* OutputOffset MUST be set to InputOffset + InputCount, rounded up to a multiple of 8.
* OutputCount MUST be set to 12.
* Flags MUST be set to zero.
* The server MUST copy a SRV_COPYCHUNK_RESPONSE, following the syntax specified in section 2.2.32.1, into the Buffer field at the OutputOffset computed above, with the following differences. ChunksWritten MUST be set ServerSideCopyMaxNumberofChunks. ChunkBytesWritten MUST be set ServerSideCopyMaxChunkSize. TotalBytesWritten MUST be set to ServerSideCopyMaxDataSize.
The response MUST be sent to the client.

Thanks,
Edgar





-----Original Message-----
From: Edgar Olougouna
Sent: Monday, December 03, 2012 2:19 PM
To: David Disseldorp
Cc: cifs-protocol at cifs.org; pfif at tridgell.net; MSSolve Case Email
Subject: RE: [REG:112120310051084] FSCTL_SRV_COPYCHUNK overlapping ranges



David,

I will investigate this and follow-up.



Thanks,

Edgar



-----Original Message-----

From: Obaid Farooqi

Sent: Monday, December 03, 2012 11:09 AM

To: David Disseldorp

Cc: cifs-protocol at cifs.org<mailto:cifs-protocol at cifs.org>; pfif at tridgell.net<mailto:pfif at tridgell.net>; MSSolve Case Email

Subject: RE:[REG:112120310051084] FSCTL_SRV_COPYCHUNK overlapping ranges



Hi David:

I have created a case to track this issue. A member of the open specification team will be in touch to further assist you soon.



Regards,

Obaid Farooqi

Escalation Engineer | Microsoft



Exceeding your expectations is my highest priority.  If you would like to provide feedback on your case you may contact my manager at nkang at Microsoft dot com





-----Original Message-----

From: David Disseldorp [mailto:ddiss at suse.de]

Sent: Monday, December 03, 2012 6:48 AM

To: Interoperability Documentation Help; cifs-protocol at cifs.org<mailto:cifs-protocol at cifs.org>; pfif at tridgell.net<mailto:pfif at tridgell.net>

Subject: FSCTL_SRV_COPYCHUNK overlapping ranges



Hi DocHelp,



FSCTL_SRV_COPYCHUNK requests may refer to overlapping file ranges. The handling of such requests is currently unclear from the documentation in [MS-SMB2] 3.3.5.15.6 Handling a Server-Side Data Copy Request.



I've outlined two such cases below, where the subsequent on disk state is dependent on specific server behaviour.



1) An FSCTL_SRV_COPYCHUNK request includes two chunks (SRV_COPYCHUNK

   records), where both specify a target range which overlaps with the

   other.



Initial State

-------------

              File:                     src                        dest

              Offset:                01234567           01234567

              Data:                   ABCDEFGH         abcdefgh



Request

-------

              FSCTL_SRV_COPYCHUNK(dest)

              SourceKey = SRV_REQUEST_RESUME_KEY(src)

              ChunkCount = 2

              Chunks[0].SourceOffset = 0

              Chunks[0].TargetOffset = 0

              Chunks[0].Length = 4

              Chunks[1].SourceOffset = 4

              Chunks[1].TargetOffset = 0

              Chunks[1].Length = 4



Resultant State

---------------

              File:                     src                        dest

              Offset:                01234567           01234567

              Data:                   ABCDEFGH         EFGHefgh



The resultant contents of dest is dependent on the order in which the chunks are processed. In the above example, Chunks[0] is written to disk before Chunks[1]. If the server where to process Chunks[1] before Chunks[0], then the resultant data would be ABCDefgh. If the server where to dispatch both both IOs simultaneously (asynchronously) the resultant data would be undefined.



Windows (tested against Server 2008 & 2012) appears to always process chunks sequentially, starting at Chunks[0] and working its way up.



Is this order of processing required, such that a chunk with a higher index must overwrite data from chunks with a lesser array index?





2) An FSCTL_SRV_COPYCHUNK request includes a single chunk, where the

   source and target ranges overlap, and the SourceKey refers to the

   same target file.



Initial State

-------------

              File:                     src_and_dest

              Offset:                0123456789

              Data:                   abcdefghij



Request

-------

              FSCTL_SRV_COPYCHUNK(src_and_dest)

              SourceKey = SRV_REQUEST_RESUME_KEY(src_and_dest)

              ChunkCount = 1

              Chunks[0].SourceOffset = 0

              Chunks[0].TargetOffset = 4

              Chunks[0].Length = 6



Resultant State

---------------

              File:                     src_and_dest

              Offset:                0123456789

              Data:                   abcdabcdef



The resultant contents of src_and_dest is dependent on the server's copy algorithm. In the above example, the server uses an IO buffer large enough to hold the entire six-byte source data before writing to TargetOffset. If the server were to use a four-byte IO buffer and started reads/writes from the lowest offset, then the two overlapping bytes in the above example would be overwritten before being read. The resultant file contents would be abcdabcdab.



Windows 2008r2 appears to use a 2048 byte copy buffer, overlapping bytes after this offset are written before being read. Windows 2012 on the other hand appears to use a buffer large enough to hold its maximum supported chunk size (1M).



Does Windows make any guarantees on what data will end up on disk when such a copychunk source/target overlap exists?



Regards, David

Microsoft is committed to protecting your privacy.  Please read the Microsoft Privacy Statement for more information.The above is an email for a support case from Microsoft Corp.REPLY ALL TO THIS MESSAGE or INCLUDE casemail at microsoft.com<mailto:casemail at microsoft.com> IN YOUR REPLY if you want your response added to the case automatically. For technical assistance, please include the Support Engineer on the TO: line. Thank you.


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.samba.org/pipermail/cifs-protocol/attachments/20121220/2abbfdf5/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: MS-SMB2 Changes for SR 112120310051084 FSCTL_SRV_COPYCHUNK overlapping ranges.pdf
Type: application/pdf
Size: 182181 bytes
Desc: MS-SMB2 Changes for SR 112120310051084 FSCTL_SRV_COPYCHUNK overlapping ranges.pdf
URL: <http://lists.samba.org/pipermail/cifs-protocol/attachments/20121220/2abbfdf5/attachment-0001.pdf>


More information about the cifs-protocol mailing list