PATCH samba cache and read-ahead

[Amin Azez]

There have been no objections, so I'll take it that there is nothing
grossly offensive in the caching yet.
I'm hoping for something of it to make to the alpha after the alpha
coming up.

The plan ahead is:

1. Support a cache-fragment map, identifying which parts of the cache as:

* empty
* unvalidated
* validated

(The current cache is linear from the beginning of the file with an
cached extent, a validated extend, and a read-ahead extent)
The cache-fragment map may be based on the tdb lock management code as
it can already cope with mapping multiple ranges of a resource, once I
work out how to do this,

Current cache behaviour will be extended to cache-fragment maps.

* If an oplock is lost or refused than all validated parts are marked as
unvalidated.
* If an oplock is held, all reads are written to the cache and marked as
validated
* If an oplock is not held, all reads are written to the cache and
marked as unvalidated.
* Reads on unvalidated parts will use the SMBreadCache operation passing
a checksum to the remote proxy so that if the cache turns out to
actually be is correct, a cheap acknowledgement can be sent.
(Unvalidated parts will only be validated if an oplock is held).

2. Writes will be written to the cache if the remote write is
successful, but I don't know how to interleave this with intermediate
async reads, advice would be appreciated.

3. add some tests

4. oplock convergence
If two satelltie users access the same file via the same satallite proxy
AND the proxy can tell, then there is no need to unvalidate the cache,
so we could might try to avoid propagating the open request and use the
same handle in order to keep the oplock.  Any advice? It may bring a low
return in exchange for the effort.

I don't see that it would help to separate the caching and compression
from the vfs_cifs module. I know someone suggested an alternate
vfs_cifs_cache module, but the caching/read-ahead is not very intrusive
to vfs_cifs and I feel can be part of the vfs_cifs module.

Compression belongs elsewhere anyway and can work between smbclient and
samba as well. (Cache-less compression of course).

I think the patch I sent lies about oplocks (because SMBclient wasn't
asking for oplocks).

Sam


* Amin Azez wrote, On 12/11/07 14:24:
> Attached is proof-of-concept caching and read-ahead for Samba to
> optimize WAN scenarios.
> It applies to Friday's SVN.
> They are distributed here under GPL3
>
> The proof of concept provides extra functionality to the cifs proxy.
>
> Read-ahead
> ========
>
> It supports configurable read-ahead.
>
> In a simple scenario where only one proxy is used on the client side of
> the WAN, if the read-ahead window is a little more than the WAN
> bandwidth multiplied by the round-trip time, then the WAN link should be
> fully utilized. On a 100ms RTT WAN link, this can reduce the time to
> load a large file by 25%; and even better when there is higher latency.
>
> Caching
> =====
>
> It supports simple caching when an op-lock is obtained, intended to
> manage better caching than the client might, but the caching is
> currently a bit simple, supporting only a linear extent from offset zero.
>
> Compression
> ========
>
> Where there is a proxy on both sides of the WAN it can also negotiate
> compression, which combined with read-ahead can reduce time to load a
> big file to 25% of normal, if the file compresses at around 25%.
>
> Cache-revalidation
> ============
>
> There is also basic cache-revalidation if there is an existing cache
> file, where the client proxy can provide a md5 checksum of what is in
> the cache as part of a read-request, and the server proxy just confirms
> this. Currently re-validation is done as compressible read-ahead, which
> means the client cannot gauge downstream bandwidth usage better than it
> can guess cache-hits, so I think I must introduce a streaming
> revalidation where the server sends unsolicited updates based on the
> compressed response size.
>
>
> A pre-cached 10MB file 25% compressible took:
>
> Normal: 470 seconds
> read-ahead: 367 seconds
> zlib + read-ahead: 97 seconds
> zlib + read-ahead+cache-validation: 9 seconds
>
> or down to around 2% of the transfer time.
>
> The code is ugly; I'm still working out which layers of Samba I should
> be tainting.
>
> Mapping generic operations
> ===================
>
> I had some trouble with cvfs; CIFS_MAP_GENERIC_DEFAULT was set to false
> yet all the handler functions were written assuming it was true, but
> also, having a union to hold the different smb_read structs turned out
> to be less helpful than it first seemed.
>
> I tried adding some extra accessor macros in interfaces.h but I find the
> whole thing unsatisfactory. I've been toying with generics of various
> kinds, including multiple invocations of the C pre-processor on
> vfs_cifs.c so we can write the function once but implement it for all
> the various smb_read structs involved. Anybody think it might be nicer
> to consider converting to vala? Me neither but I don't like managing
> this hard work be hand. Passing round a union* in this fashion is as bad
> as passing round a void* except we don't get the compiler warnings. I
> think the void* would make me feel happier as we could cast it with a
> protective macro that checked the level, what we have now is plain
> dangerous.
>
> New operations
> ==========
>
> I defined a SAMBA wire-protocol operation SMBreadMagic but it's
> currently really just SMBreadx with compression support added. I'm
> agreed that a new operation needed adding here, but I'm not totally
> certain that I needed a new samba-level operation SMB_READ_CACHE. It's
> so similar to SMB_READ_READX that the code that handles it is nearly the
> same, only can't quite be because they use different members of the
> smb_read union (bah!) and I don't think much of my accessor macros.
>
> Advertising capabilities
> ================
>
> I also took the liberty of:
> #define CAP_CACHE              0x00100000
>
> so client and server could tell if they supported the caching and
> compression stuff. I'm not sure about this being the best way, I don't
> feel good stealing one of these bits, maybe we need an
> extra_capabilities field to use when talking to samba servers?
>
> smb.conf
> ======
>
> The smb.conf file has to look something like this:
>
> [myshare-8]
>         ntvfs handler = cifs
>         cifs:server = 10.0.0.5
>         cifs:share = myshare
>         cifs:cache-enabled = FALSE
>         cifs:cache-readahead = 8192
>         # I think these next two are not used, now we have MAGIC_CA
>         cifs:server-compression = FALSE
>         cifs:client-compression = TRUE
>
>         cifs:user = USERNAME
>         cifs:password = PASSWORD
>         cifs:domain = DOMAIN
>
>
> I'm now trying to find out how to get the cifs-proxy to proxy the
> connecting users credentials. Any help would be appreciated.
>
> Sam
>