plan for a feature: fsync for assuring file size

[James Peach]

On Oct 24, 2007, at 11:21 AM, SER.RI-TIC - David Losada wrote:

> Hello,
>
> some time ago we had a problem that was causing silent data loss in  
> our
> users' files. It was a problem related to Linux behavior of deferring
> NFS quota errors until the file is closed or fsync'ed.
>
>    http://www.spinics.net/lists/samba/msg67798.html
>
> With Jeremy's patch and configuring Samba to fsync after _every_  
> write,
> the data loss problem was stopped. That came with the price of a big
> performance hit, of course. We need to decrease this hit, and here I'm
> sharing with you my thoughts of how to do it.
>
> I'd like to gain some insight from you about (a) how I can implement  
> it
> correctly in Samba's architecture and (b) if this feature could be  
> added
> into the standard Samba distribution.
>
> Our aim is to be able to send an NT_STATUS_DISK_FULL message to a
> Windows client *before* it asks to close the file. I have thought of
> this in the context of non-sparse files (strict allocate = yes), maybe
> some ideas can be applied to the case of sparse files.  The feature
> would be user-enabled, at share level.
>
> There's a straightforward idea, and a slightly smarter one.
>
> 1) *straightforward*: performing fsync() after allocating file size
> [maybe at vfs level in smbd/vfs-wrap.c:strict_allocate_ftruncate?]  
> This
> way fsync is just called once per file. Should help performance a
> little, even if it means some dumb effort by forcing writing zeros to
> disk/network.
>
> 2) *bit smarter*: performing fsync() when writing the end of the file.
> The fsync() call could be deferred until the moment when the client  
> asks
> to write the _last byte_ of the file. My impression is that this  
> should
> be done above VFS level (smbd/fileio.c maybe?), because it would  
> enable
> it to work together with write caches.

If you care about your data, you don't want to use the write cache  
feature. Not that it's a bad feature, but it can really open the  
window for data loss in the event of a power failure.

Take a look at the commit module. It does a fsync at file close time  
and at configurable strides through the file.

http://samba.org/samba/docs/man/manpages-3/vfs_commit.8.html

For example, to fsync after every 10K worth of data is writen, do this:

[share]
	vfs modules = commit
	commit:dthresh = 10K

--
James Peach | jpeach at samba.org