threads and VMS [Re: dce/rpc "client" api]

[Andrew Tridgell]

Bob,

I'm not convinced that multi-threading smbd would actually give a
performance improvement, even for artificial things like netbench.
Although on the surface it might seem obvious it is not at all obvious
when you dig a bit deeper.

Here's why:

- only two types of operations take enough time that it would be worth
  trying to process another SMB while waiting. They are reads and
  blocking locks. (I'm ignoring printing here for now, see my earlier
  email about printing and threads). Note in particular that I do not
  include writes in this list, that's because a modern OS returns on a
  write pretty much immediately. All the kernel does is copy the data
  into the page cache and then the actual disk IO happens
  asynchronously.

- for blocking locks smbd currently pushes the lock request onto a
  list of pending blocking locks and continues processing with the
  next packet on the socket (see push_blocking_lock_request()). So for
  those we are already "multi-threaded".

- for reads we are not multi-threaded. Does it matter? If the data is
  coming from the page cache then probably not (unless our network
  bandwidth is nearly as large as our memory bandwidth). If the data
  is coming from disk then we could gain something if there are other
  requests pending on the same socket that could be satisfied without
  a disk read, or with a disk read on a different disk spindle.

- the above all assumes a single client. Now what happens when we
  are serving a dozen or (as happens in a netbench run) hundreds of
  clients? You only gain when you multi-thread if your your single
  threaded system doesn't have anything useful to do while waiting for
  a read to complete. When there are large numbers of clients this is
  unlikely.

It may be that adding something like push_blocking_read_request()
combined with aio_read() (which is how we'd have to implement IO
threading in Samba) is a worthwhile addition, but I don't think it is
obvious that it will be a win. If anyone wants to investiage this then
start by writing a simulator (use tbench as a basis if you like - it
simulates the current Samba code) and write some tests against the
simulator to see if we really gain anything.

Cheers, Tridge