Patchset to add asynchronous open/close to master

[Stefan (metze) Metzmacher]

Hi Jeremy,

> So here is something I've been working on for an OEM
> that I've like to get reviewed and pushed into master.
> 
> It adds the ability to do asynchronous creates and closes
> to master for both SMB1 and SMB2 clients. This is a *massive*
> (factor of 10x) help in situations where there is a slow(ish)
> clustered filesystem and there is one multithreaded Windows client
> process doing lots and lots of file creates. Yes, horrible
> applications like this exist :-).
> 
> Voker was a massive help to me in bouncing the ideas
> off him for this change, and coming up with ways to
> solve the race conditions it might have created (but
> now does not :-).
> 
> It's also a model for how we can add more thread-based
> parallelism into the file server without large rewrites.
> 
> There are only a few small changes to the core fileserver
> code, most of the new code is inside vfs_aio_pthread.c.
> 
> Michael, this is a perfect example of the kind of new
> feature that OEMs want to be able to add to 3.6.next
> without waiting for a full 4.0.x release.
> 
> It was actually developed in 3.6.x and forward ported
> to master - and it's a total of only 1352 lines of
> diff (documentation and build system updates included).
> 
> I hope this helps understand the kind of thing I'm
> talking about when I mention feature addition to
> 3.6.x, and would be covered nicely by Ira's suggestion.
> 
> Anyway - enjoy, and if I don't hear anything back I'll
> start trickling the changes into master over the next
> week or so (it's actually a fairly simple patchset).

It may look simple, but it isn't!

From the manpage "These are per-share parameters", this is wrong!

The usage of openat() let the module skip the next SMB_VFS module!

There's no infrastructure to cancel async opens! I think it can happen
that the client believes the open was canceled, but the file gets created.

find_completed_open() may checks !(opd->flags & O_EXCL), shouldn't
check O_CREATE? as aio_open_fn()

aio_close_worker() uses 'close' and skips the next SMB_VFS module!

There's no infrastructure to defer async close calls,
so the smb layer calls file_free(), which means we return the result to
the client before the close is done.

adding 'uint64_t mid' to files_struct is an incompatible change.

Just checking if (st.st_uid == opd->uid) isn't enough,
the global context might be switched to a force group context.

Instead of extending the usage of the deferred_open_queue,
we should really create a generic infrastructure based on
tevent_req based _send/_recv function pairs.

metze

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 262 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20120613/b039fae7/attachment.pgp>