[linux-cifs-client] ENOSPC without O_SYNC

[Oliver Martin]

Am Fri, 18 Jul 2008 17:12:53 -0400 schrieb Jeff Layton:

> > > could mount another share. It's supposed to stop when it runs out
> > > of space, but it kept copying data right into nowhere.
> > 
> > What's supposed to stop? To quote the write(2) manpage:

I meant dar, the backup program.

> > A  successful return from write() does not make any guarantee that
> > data has been committed to disk.  In fact, on some buggy
> > implementations, it does  not  even guarantee that space has
> > successfully been reserved for the data.  The only way to be sure
> > is to call fsync(2)  after  you  are done writing all your data.

So it explicitly calls implementations which do not fail if not enough
space is available buggy?

> > ...if you're not calling fsync() then there are no consistency
> > guarantees here.

The write manpage also says:
POSIX  requires  that  a  read(2)  which can be proved to occur after a
write() has returned returns the new data.  Note that not all file sys‐
tems are POSIX conforming.

Now the questions is, does cifs strive to be POSIX compliant in this
respect? If it does, then the current behaviour for async writes is a
problem, because if writes succeed when there is no space available, at
some point data has to be dropped. The section of POSIX referred to
explicitly mentions caching schemes for networked file systems as a
particularly tricky situation.

[...]

> > fusesmb is probably doing synchronous writes.

Could be. Throughput is about the same with fusesmb and cifs with
directio, slightly less than 7 MB/s. With buffering it is around 10
MB/s.

> > 
[...]
> > 
> > What might be nice is at some point to implement the solution that
> > nfs did a year or so ago and make it so that when writes to a file
> > start failing cifs would flip to using synchronous writes. That's
> > really just a "nice to have" though.

I think that would be equally dangerous for a program that assumes a
POSIX compliant file system: Suppose I have a program that writes data
to a file until it receives ENOSPC, and then opens another file on a
different file system. There would be some data I thought was written
successfully, but in reality was not, which would therefore be lost.
POSIX guarantees that, in absence of a system crash before it was
actually written to disk, data can later be read after write succeeded.
So I think it should be okay to do this without calling fsync.

How expensive would it be to synchronously reserve space on the server
during write? I guess it would slow down applications doing lots of
small writes, while it wouldn't matter much for those writing large
amounts of data, because after the buffer is full, they are limited by
network throughput.
Of course, maybe the best option is just to document that cifs is only
POSIX compliant when mounted with -o directio.


Regards,
Oliver