sparse file handling

[David Disseldorp]

Hi Björn,

On Wed, 13 Jan 2016 12:45:14 +0100, Björn JACKE wrote:

> Hi David,
> 
> I just had a look at sparse file handling and SEEK_HOLE/SEEK_DATA support in
> Linux and I stumbled over some things here.
> 
> 43e581188ab10 adds the FILE_SUPPORTS_SPARSE_FILES capability in case that
> HAVE_FALLOC_FL_PUNCH_HOLE HAVE_LSEEK_HOLE_DATA are defined at compile time.
> This is done unconditionally of the filesystem that is actually being used.

Indeed.

> On the one hand we obviously do have sparse file support on any POSIX
> filesystem and on the other hand the *filesystem* might not support SEEK_HOLE
> or PUNCH_HOLE even though the kernel *generally* has support for that.
> 
> This is why I think the above generic condition of sparse support is not ideal.
> Better might be to assume sparse capability generally and in sparse file
> related functions react accordingly if lseek is giving back ENXIO like in cases
> where SEEK_HOLE/SEEK_DATA is unsupported or if the tried fallocate call fails.

The FS capability needs to be known prior to any FS IO, so this approach
sounds a little suboptimal to me. Besides, the capability flag can be
manually set via share:fake_fscaps.

> In the case sparse file capability is not set, SMB2 returns
> NT_STATUS_NOT_SUPPORTED while SMB1 dummy-returns the whole range to work around
> a bug, see 15531d9df1 and the comment on that in 3fdebc06465c.

Actually, if the sparse file attribute isn't set then SMB2+ QAR responds
indicating that the whole range is allocated (like SMB1). It's only when
the sparse file attribute is set and HAVE_LSEEK_HOLE_DATA is undefined
(in which case FILE_SUPPORTS_SPARSE_FILES=0) that it returns
NT_STATUS_NOT_SUPPORTED.

> smb2 should probably do the same as smb1 here to make MKS cp (and possible more
> software) not break only on smb1 and work on smb1.
> 
> It would also be nice to get the pieces from fsctl_qar_seek_fill() from
> source3/smbd/smb2_ioctl_filesys.c be more available so that this can be used in
> source3/modules/vfs_default.c for FSCTL_QUERY_ALLOCATED_RANGES also.

I agree that SMB1 and SMB2+ should be consistent here, preferably with as
much code shared between the two as possible.
One (untested) option may be something like:

index 6e4a785..2b4002f 100644
--- a/source3/smbd/smb2_ioctl_filesys.c
+++ b/source3/smbd/smb2_ioctl_filesys.c
@@ -392,7 +392,8 @@ static NTSTATUS fsctl_qar(TALLOC_CTX *mem_ctx,
        max_off = MIN(sbuf.st_ex_size,
                      qar_req.buf.file_off + qar_req.buf.len) - 1;
 
-       if (!fsp->is_sparse) {
+       if (!(fsp->conn->fs_capabilities & FILE_SUPPORTS_SPARSE_FILES)
+                                                       || !fsp->is_sparse) {
                struct file_alloced_range_buf qar_buf;
 
                /* file is non-sparse, claim file_off->max_off is allocated */

Alternatively, we could return an NT_STATUS_NOT_SUPPORTED error for
FSCTL_SET_SPARSE(set_sparse=true) requests when
FILE_SUPPORTS_SPARSE_FILES is zero, but that would mean changing long
standing server behaviour.

Best option would be to run some tests against a Windows server
with/without FILE_SUPPORTS_SPARSE_FILES, perhaps with FAT or an old ReFS
backed share.

Cheers, David