Samba + exFAT : how to avoid pre-allocating when copying big files?
namjae.jeon at samsung.com
Tue Dec 8 03:18:43 UTC 2020
> -----Original Message-----
> From: Richard Sharpe <realrichardsharpe at gmail.com>
> Sent: Tuesday, December 8, 2020 11:21 AM
> To: Namjae Jeon <namjae.jeon at samsung.com>
> Cc: Joseph <j at gget.it>; samba-technical <samba-technical at lists.samba.org>
> Subject: Re: Samba + exFAT : how to avoid pre-allocating when copying big files?
> On Mon, Dec 7, 2020 at 6:12 PM Namjae Jeon via samba-technical <samba-technical at lists.samba.org> wrote:
> > > Hello,
> > > When sending a big file (for example 2 GB) from a Windows computer
> > > to a Linux computer (with Samba server), then:
> > > * if the destination FS is ext4, NTFS, no problem
> > > * if the destination FS is exfat, then there is a strange delay (~30
> > > seconds) during which the Windows Explorer waits, before actually
> > > copying the file It seems that the Samba server first "preallocates" the 2GB, before accepting to
> start the copy from Samba client.
> > > Thus, 2GB (of probably null bytes?) are first written, and then, the actual file is written.
> > > How to avoid this effect that doubles the number of gigabytes written and doubles the transfer
> > >
> > > I have already tried:
> > >
> > > strict allocate = no
> > >
> > > without any success in the exfat case.
> > >
> > > Any idea?
> > Windows seems to send SMB_FILE_END_OF_FILE_INFORMATION smb2 setinfo
> > request to pre-allocate space as much as the file size before writing the file.
> I believe that Windows does this because applications are actually writing into the Windows equivalent
> of the page cache.
> To avoid problems where the server runs out of space when the cache flushes, they send a request to
> extend the file to the new EOF when a write to a larger offset occurs. That way, the ENOSPC error can
> be returned on the WRITE rather than on the CLOSE because no one checks the error return from a close
> and anyway, how can you tell which write failed when you get an error on close?
I didn't want to remove ftruncate before writing in samba. I mean that there is
an improvement if exfat fallocate is implemented. One misunderstanding is that
the FileEndOfFileInformation request must change the size of the file.
Because of that, We may not use keep size flags of fallocate(). BTW, Why do
windows call FileEndOfFileInformation instead of FileAllocationInformation
of smb2 setinfo before writing?
> > samba fallback to call ftruncate if fallocate is not support in local filesystem.
> > I personally think that it is possible to improve the problem if exFAT
> > support keep size flags of fallocate(). It doesn't support yet in
> > linux kernel exFAT, but I have a plan. exFAT can store the number of
> > allocated cluster size and the current file size in metadata of file.
> > Unlike FAT. so it seems that the fallocate can be implemented properly in exFAT.
> > >
> > > PS:
> > > * I've already tried exfat-fuse, but also exfat-nofuse, and also
> > > exfat (non
> > > fuse) included in Kernel 5.7, and it's the same.
> > > * There are many forums posts with the same problem, e.g.
> > > https://www.raspberrypi.org/forums/viewtopic.php?t=206866
> Richard Sharpe
More information about the samba-technical