Samba with XFS and atomic read/write guaranty

[Jeremy Allison]

On Mon, Mar 25, 2019 at 03:12:14PM +0200, Amir Goldstein via samba-technical wrote:
> Hi Volker,
> 
> As we discussed offline, we have been observing > 200ms
> latencies on read/write operations when running a random
> read/write workload on single file over samba on an xfs filesystem.
> [aio read size = 1; aio write size = 1; aio max threads = 32]
> These latencies are not observed when running the same workload
> over samba with ext4 filesystem.
> 
> I finally got around to investigating the difference between xfs
> and ext4. The results of my investigation are unfolding in this
> fsdevel thread:
> https://marc.info/?l=linux-fsdevel&m=155350018822848&w=2
> 
> It turns out that xfs respects POSIX w.r.t "atomic read/write" and
> this is implemented by taking a file-wide shared lock on every
> buffered read.
> This behavior is unique to XFS on Linux and is not optional.
> Other Linux filesystems only guaranty page level atomicity for
> buffered read/write.
> 
> I have proposed to make the xfs "atomic buffered read/write"
> behavior optional and will see how that proposal plays out, but
> in the mean while, could you say if SMB has a requirement for
> atomic read/write and/or if samba handles the serialization
> for read/write to overlapping file areas itself?
> 
> To put it another way, to the best of your knowledge, does the
> fact that ext4 does not guaranty atomic read/write (for entire IO
> buffer) or the fact that xfs might loose this guaranty have any
> negative affect on samba users?

That is a fascinating thread, thanks a *LOT* for posting it.

Samba does not handle the serialization for read/write to
overlapping file areas itself.

If a client does it it gets what it gets back from the
buffer cache. Well-written client applications don't
do this, as (IMHO) they can't guarantee consistent
behavior from the file system.

In other words, XFS is proving more safety semantics
than Samba (or any client to my knowledge) depend on.

Or - yes, you're right here :-).

Jeremy.