Ceph RADOS linearizable?

[David Disseldorp]

Hi Ralph!

This a question better suited to the Ceph development list, but I'll do
my best to try to answer...

On Wed, 8 Mar 2023 15:53:31 +0100, Ralph Boehme wrote:

> Hey David,
> 
> I'm currently doing some research into distributed databases. I'm 
> tracking candidates here:
> 
> https://wiki.samba.org/index.php/Samba_on_Scale
> 
> One of the tricky things is figuring out the consistency level [1] 
> provided by the database. There's no public RADOS documentation that 
> clearly describes the consistency provided in industry standard terms, 
> I'm pretty sure it's linearizabile, but the only real reference 
> mentioning linearizability I could find is from a RADOS related research 
> paper that talks about adding support for weaker consistency levels:
> 
> http://www.cs.nthu.edu.tw/~ychung/conference/2016_ICPADS.pdf
> 
> Can you confirm whether RADOS is indeed Linearizabile? I'm pretty sure 
> it is, but would like to be sure. :)

RADOS is a very broad interface when considering linearizability, but
if you choose to focus on key/value storage accessed via the Ceph omap
interface, then yes, my understanding is that OSD requests for a single
object are processed in a way that provides atomic consistency from a
RADOS client perspective. That guarantee goes out the window if multiple
objects are involved, or the I/O is split across multiple OSD requests;
a single OSD request can contain multiple (sequentially handled) I/O
operations.

Cheers, David