FAT, NTFS, CIFS and DOS attributes

[Andrew Tridgell]

Steve,

 > Can you really create NTFS files (in that other OS) with the sparse
 > flag (does it really do anything differently if you create with
 > that flag)?

yes

By default files created on NTFS are not sparse. You can mark them
sparse using a FSCTL_SET_SPARSE ntioctl() call. See the
torture_set_sparse() function in Samba4 smbtorture code for an example
of how to do this remotely via CIFS (its an nttrans call).

This is actually a significant problem for Samba which we have
historically just ignored. Applications on NT can create a file, then
use the ALLOCATION_INFO or END_OF_FILE_INFO levels of the SetFileInfo
call to force NTFS to allocate space. Then the application can start
writing while being absolutely sure that it won't get an out of space
error during a write within the allocated space. 

In Samba3 what we do is just succeed the FSCTL_SET_SPARSE ntioctl, but
not actually do anything. We also pretend to succeed ALLOCATION_INFO
calls, and using END_OF_FILE_INFO beyond the end of file just uses
ftruncate(), which (incorrectly in most cases) leaves it sparse.

In Samba4 we also pretend to succeed the FSCTL_SET_SPARSE call, but we
also store the allocation size given in the ALLOCATION_INFO call (in
the user.DosAttrib xattr), and return it when queried (as long as
later writes haven't invalidated it).

What I'd really like is for some way to tell a unix filesystem not to
make a file sparse, so that we can implement NTFS semantics correctly
for applications that rely on it. I'd like to do this without having
to actually call pwrite() with lots of zeros.

There are higher priority things that I'd ask for from the kernel for
better Samba support, but this would definately be in the top 10 :-)

Cheers, Tridge