What's next for Samba ?

[tridge at samba.org]

Jerry,

This is probably a good time to give the status of the Samba4 effort,
especially as I think the code has reached the point where a wider
pool of programmers is appropriate.

The Samba4 effort started nearly a year ago, originally driven by the
idea of allowing for non-POSIX filesystem backends. Since then the
scope of the project has grown considerably, and I think it would be
fair to describe it now as an effort to "build a better Samba". By
"better" I mean:
 
  * more room to grow in terms of possible features, due to a much
    more flexible architecture
  * a much more maintainable code base, using programming techniques
    that are more robust and less bug-prone
  * a much more complete implementation of CIFS. It implements all of
    the nasty little corners of the protocol that we have ignored
    before
  * a highly testable architecture, with the idea of being able to
    properly test every function built into the design

Here ends the advertising section ....

We really have two major ways forward from here. We can either
backport some of the more eligible pieces of Samba4 into Samba3, or we
can concentrate on getting Samba4 complete enough to be really
used. Or we can do both (and yes, I know that means I can't count).

In terms of individual modules the parts of Samba4 that would be
obvious candidates for back-porting are:

 * the libcli/ SMB client library. The client library in Samba4 is
   vastly better than the one in Samba3. It is *much* more complete in
   terms of protocol coverage, it is asynchronous, it is (mostly)
   thread-safe and reentrant and it is written in a much more
   maintainable fashion. There are still some areas of it that could
   be improved, and there are one or two features in the Samba3 libsmb
   that haven't been ported yet, but I think it is an obvious
   candidate for backporting. The downside would be that we would need
   to make quite significant changes to Samba3 to incorporate it.

 * the event handling code. I think this is vastly better in design
   to the Samba3 select loop. It could be backported quite easily I
   think.

 * the new librpc/ library that I have been developing recently. I
   already have a lot more of the LSA pipe done than was in the old
   rpc_client library, and adding new functions by writing IDL is
   *much* faster than the old way. The new code is also much more
   accurate (I have found numerous bugs and omissions in the old
   handwritten rpc code) and has the advantage that writing a test
   suite is trivial, as most of the testsuite code is auto-generated
   by the IDL compiler for us. The new code is also much less prone to
   security problems, as all of the necessary parsing checks are
   auto-generated. I think that we will be able to produce a fully IDL
   based rpc client and server subsystem quite fast. Trying to
   backport this to Samba3 in any meaningful way would be a lot of
   work, but is something that should be considered.

The question is, should we backport at all? Past experience
(especially with Samba3) tells us that backporting will lead to a long
delay before Samba4 is out, as we keep extending the life of
Samba3. It would also lead to the possibility of eventually abandoning
Samba4 and instead just using Samba3 with some bits of Samba4 tacked
on. While I think that would be better than Samba3 as it is now, I
also think that we won't see nearly as much benefit in terms of code
maintainability if we do this, as we will be left with the core
infrastructure of Samba3 which I think is really holding us back.

I'd also appreciate some volunteers to work with me in the short term
on some core Samba4 features that need developing. In particular:

 * the substitution system is completely missing. We need a way of
   handling all those % substitutions from Samba3, and I think that to
   do that we will need to add a context handle to all lp_*()
   calls. That is a lot of work, but I hope someone will take it on.

 * someone needs to start on the POSIX NTVFS backend. That is a big
   job, particularly if done right (to do it right it will need to be
   async). I'd suggest writing a synchronous one first, then
   considering how to build a async posix backend once we have a
   synchronous one done.

 * I'd really like some help writing all the IDL files for
   librpc/idl/. That will almost certainly involve extending the IDL
   compiler to handle some constructs that it doesn't currently
   handle. I'll need to work with whoever does this to see how the
   extensions can be done (for example, I'm working on fixing
   conformant arrays in structures at the moment)

 * I'd like some help in adding some features to librpc, such as
   schannel support

 * I'd like some help in writing the server side infrastructure for
   Samba4 rpc, using auto-generated stubs

 * and lots of other stuff .....

I'd also appreciate it if anyone working on Samba4 first read the
prog_guide.txt in the Samba4 cvs tree. It isn't complete yet, but it
should give you a bit of an idea of the style that I am aiming for.

Cheers, Tridge