Duplicate SMB file_ids leading to Windows client cache poisoning

[Steven Engelhardt]

Samba Team,

While trying to roll out recent versions of Samba, we believe we have discovered a bug in Samba related to SMB file id generation which can lead to Windows SMB clients to observe wrong data.

Samba version 4.11.1 changed how it generates SMB file ids to be instantiation time-based. This change introduced a bug in that it made it possible for two different files on the same volume to have the same file id, in volation of the SMB spec.  This appears to lead to cache poisoning in the Windows SMB client-side cache, in which two different files share a common cache key, which can cause Windows SMB clients to observe wrong data (e.g. the length of a file is reported incorrectly).

We first noticed this with a C# client program on Windows which created 10,000 files in parallel, each of a different file size (e.g. the first file has size 1 byte, the second has size 2 bytes, etc.), and then checked the file sizes of each of the files one-by-one using Directory.EnumerateFiles() and FileInfo.Length, noting any time that a file does not have the correct expected size.
 
This bug is particularly easy to trigger on Ubuntu 18.04 in WSL2 on Windows 10, as it appears to have an effective itime clock resolution (based on observing returned st_ctime and st_mtime from `stat(2)` system calls) of 10ms.  In other words, if you can create two files on a WSL2 machine within 10ms of each other, its exceptionally likely that they will be assigned the same itime and hence have the same SMB file id.  However, we have also observed this bug in the wild on Ubuntu 16.04 VMs and Ubuntu 18.04 VMs running in Azure.  Obviously, the better resolution of the clock from stat(2), the less likely this bug is to trigger.

We tested a number of different Samba versions and observed the following:
| Samba Version | Date Tested | Status |
| ------------- | ----------- | ------ |
| 4.10.18       | 2021-11-26  | Good   |
| 4.11.0        | 2021-11-26  | Good   |
| 4.11.1        | 2021-11-26  | Bad    |
| 4.11.3        | 2021-11-26  | Bad    |
| 4.11.7        | 2021-11-26  | Bad    |
| 4.11.17       | 2021-11-26  | Bad    |
| 4.13.10       | 2021-11-26  | Bad    |
| 4.13.11       | 2021-11-26  | Bad    |
| 4.13.14       | 2021-11-26  | Bad    |
| 4.14.10       | 2021-11-26  | Bad    |
| 4.15.2        | 2021-11-26  | Bad    |

Using git bisect, we traced the bug to the commit 459acf2728aa0c3bc935227998cdc59ead5a2e7c.

We can also deliberately trigger the bug by creating multiple files in a directory with identical itimes by manually setting the user.DOSATTRIB xattr, and observe that these files are served with duplicate SMB file_ids, in violation of the SMB protocol.

We wrote a simple patch to temporarily remove itime-based file id generation which has resolved this bug for us:

diff --color -Naur samba-4.13.11.orig/source3/modules/vfs_default.c samba-4.13.11/source3/modules/vfs_default.c
--- samba-4.13.11.orig/source3/modules/vfs_default.c	2021-08-09 02:20:55.000000000 -0500
+++ samba-4.13.11/source3/modules/vfs_default.c	2021-11-30 15:40:00.089302200 -0600
@@ -3003,10 +3003,6 @@
 {
 	uint64_t file_id;
 
-	if (!(psbuf->st_ex_iflags & ST_EX_IFLAG_CALCULATED_FILE_ID)) {
-		return psbuf->st_ex_file_id;
-	}
-
 	if (handle->conn->base_share_dev == psbuf->st_ex_dev) {
 		return (uint64_t)psbuf->st_ex_ino;
 	}

We would appreciate any guidance on the correct long-term resolution of this issue.

Steven Engelhardt
Relativity