[cifs-protocol] 119121124001994 [MS-FSA] Setting end-of-file on file-handle with pending delayed write-time update

Wed Dec 18 18:22:50 UTC 2019

Hello Ralph, below are the answers for your questions from the  product team.

As the legacy delayed write-time handling is not (was never?) documented in MS-FSA, I'm asking for confirmation that the following conclusion is correct:

* a setinfo-eof on a file-handle flushes pending
  write-time updates

[Answer]  Correct.  I would avoid using the term "flush" though because that has additional implications.  For example the single byte that was written earlier as a cached Write is not flushed to disk.  In the scenario described, when the cached Write occurred, this was "noted" (i.e. a flag was set) in a per-handle structure.  If nothing else exciting were to happen then at Cleanup time this note would be consumed and the last write time would be updated to the time of the Cleanup.  But sometimes there are opportune moments where we are going to update some fields of the file on disk, for example file sizes, where it's convenient to see what other fields of the file we can update at the same time so as to batch it.  That's what's happening here.  Since we are going to update the EOF on disk, we also check for these "notes" of timestamps that need updating and consume them.  This saves us from updating the timestamps again at Cleanup time again unless further writes occurred in the meantime.

These are all heuristics used to improve perf - the delaying of updating timestamps at all, and the batching of timestamp + file size + file attributes updates together when possible.

The abstract model used in MS-FSA models all timestamp updates happening immediately, which is a perfectly valid model (indeed the best model from a correctness point of view).  Real world implementations are free to make correctness vs performance tradeoffs as desired.  That precedent has been set.  But the specifics vary from NTFS version to version, and vary wildly from NTFS vs other file systems.  I would say the only expectation is that timestamps get updated sometime between when an operation was performed and when the handle performing the operation gets closed (Cleanup).  This could never realistically get done after Cleanup because structures get torn down and state gets lost.

* flush happens even if the requested size equals the existing size

[Answer]  Correct.

If this is correct, are there other operations that trigger this behaviour? please update the spec as appropriate

I have scanned through the current source code as of Windows 10 1909 and the operations where the "notes" to update timestamps get consumed are the following:

Cleanup
SetBasicInfo
SetAllocationInfo
SetEndOfFileInfo
SetValidDataLengthInfo
Flush
FSCTL_SET_ENCRYPTION
FSCTL_OFFLOAD_WRITE

Remember MS-FSA is an abstract model describing "a" correct implementation, based on NTFS but simplified somewhat.  To represent it exactly it would essentially be the source code.  The source code incidentally can be licensed.  If you do have the source code, look for places where NtfsUpdateScbFromFileObject is called.  That's where the "notes" are consumed.

Also chapter 6 of MS-FSBO (File Systems Behavior Overview) has some good information about timestamps, written in the Windows 7 timeframe:

http://download.microsoft.com/download/4/3/8/43889780-8d45-4b2e-9d3a-c696a890309f/File%20System%20Behavior%20Overview.pdf

-----Original Message-----
From: Sreekanth Nadendla 
Sent: Wednesday, December 11, 2019 9:28 AM
To: Ralph Boehme <slow at samba.org>
Cc: cifs-protocol at lists.samba.org; support <support at mail.support.microsoft.com>
Subject: 119121124001994 [MS-FSA] Setting end-of-file on file-handle with pending delayed write-time update

Dochelp in Bcc
Casemail in Cc

Hello Ralph, I will be assisting you with your question. I am currently researching the problem and will provide you with an update soon. Thank you for your patience.

Regards,
Sreekanth Nadendla
Microsoft Windows Open Specifications

-----Original Message-----
From: Ralph Boehme <slow at samba.org> 
Sent: Wednesday, December 11, 2019 1:50 AM
To: Interoperability Documentation Help <dochelp at microsoft.com>
Cc: cifs-protocol at lists.samba.org
Subject: [EXTERNAL] [MS-FSA] Setting end-of-file on file-handle with pending delayed write-time update

Hello dochelp!

Note: the following only applies to Windows versions where the SMB server implements delayed write-time updates. I've verified that the behaviour described below is reproducible against the following Windows
versions: Windows 7, Windows 2008r2 and Windows 2016.

With Windows 2019 I see a different behaviour, which is as expected, as it implements immediate timestamp updates.

SMB2 scenario:

1) create file -> file handle H1

2) sleep one second

3) write one byte to H1

4) getinfo on H1 -> write-time hasn't been updated, delayed update
   still pending

5) setinfo end-of-file with size=1 on H1
   (so existing size = requested new size)

6) getinfo again on H1 -> now the write-time has been updated

This is the behaviour that is reproducible against Windows 7, Windows 2008r2, Windows 2016, but not Windows 2019.

Apparently, the setinfo in step 5 flushed the pending write-time update.

Question: as the legacy delayed write-time handling is not (was never?) documented in MS-FSA, I'm asking for confirmation that the following conclusion is correct:

* a setinfo-eof on a file-handle flushes pending
  write-time updates

* flush happens even if the requested size equals the existing size

If this is correct, are there other operations that trigger this behaviour?

pcap: <https://nam06.safelinks.protection.outlook.com/?url=http:%2F%2Fwww.samba.org%2F~slow%2Fpcaps%2Fbug14150.pcapng.gz&data=02%7C01%7Csrenaden%40microsoft.com%7Cbc697f25c8234a79223708d77e06660b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637116438260754970&sdata=4xPCL7XindNk%2B%2FSS4HHa5fmi12Mga8g4Y6KAz5TkfsI%3D&reserved=0>

For completenes, here's also a pcap against Windows 2019:
<https://nam06.safelinks.protection.outlook.com/?url=http:%2F%2Fwww.samba.org%2F~slow%2Fpcaps%2Fbug14150-w2019.pcapng.gz&data=02%7C01%7Csrenaden%40microsoft.com%7Cbc697f25c8234a79223708d77e06660b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637116438260754970&sdata=5wWjHnaLWVy0Hqykl%2Fyz0dBxglt3VBJlRnfHyzJsIlE%3D&reserved=0>

This one stops at step 4 when the client test driver detects that write-time did already change.

Thanks!
-slow

-- 
Ralph Boehme, Samba Team                https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsamba.org%2F&data=02%7C01%7Csrenaden%40microsoft.com%7Cbc697f25c8234a79223708d77e06660b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637116438260764964&sdata=mXCjMztDUwhKFSxcH%2BSoreyDXoXNCbtzmpakBzDxyfM%3D&reserved=0
Samba Developer, SerNet GmbH   https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fsernet.de%2Fen%2Fsamba%2F&data=02%7C01%7Csrenaden%40microsoft.com%7Cbc697f25c8234a79223708d77e06660b%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637116438260764964&sdata=HiPThHlFH25RgCvk%2BPTnPYMKcek%2FJPGrZNbBpt88LEs%3D&reserved=0
GPG-Fingerprint   FAE2C6088A24252051C559E4AA1E9B7126399E46