Samba + exFAT : how to avoid pre-allocating when copying big files?

[Joseph]

Thanks Ronnie for your answer.

I've used NTFS before (ntfs-3g which uses FUSE), but it's slow on
RaspberryPi (20-30 MB/s) whereas exfat can reach 70 MB/s.
One solution would be to wait for the new ParagonSoftware open-source NTFS3
driver which will (or not?) be merged in Linux kernel.
But I think it won't be easily available on RaspberryPi soon.

So I decided to try with kernel's exfat (non-fuse) which is very fast ...
except this problem.

> Try adding a f.truncate(...) to set the file size after you open the
> file but before you f.seek()

You're right Ronnie: a f.truncate(1000*1000*1000) here takes 16 seconds,
i.e. writing 1 GB of null bytes at 60MB/s, that sounds correct.

--

This is confirmed by Jeremy's analysis (via email), here is a summary
obtained after looking at my logs where it gets stuck for 30+ seconds:

>  smbd_do_setfilepathinfo: test/a.rar (fnum 1649140843) info_level=1020
totdata=8

-> that's an SMB2 SMB_FILE_END_OF_FILE_INFORMATION call.
-> This is going into (ultimately) vfs_set_filelen().
->  SMB_VFS_FTRUNCATE() call to set the length.
-> static int vfswrap_ftruncate(vfs_handle_struct *handle, files_struct
*fsp, off_t len)
probably here?
https://github.com/avati/samba/blob/master/source3/modules/vfs_default.c#L1813

So two possibilities:

* is there a way to set an EOF on a file descriptor on exFAT that *doesn't*
do the allocation? This would require a modification in the exfat driver?

* would it be possible to have a mode in Samba in which it never
"truncates"?

    [global]
    no_truncate = yes

The file size would grow when new data is appended when a file is copied
(like my code in Python before, with only f.write(...)), but no truncate at
all.

Do you think this would be possible?

Here is a linked report on the Github page of exfat-fuse:
https://github.com/relan/exfat/issues/45