How to move storage OEMs to Samba 4.0 ?

[Michael Adam]

Hi Jeremy,

Jeremy Allison wrote:
> Hi all,
> 
> So the full Samba 4.0 release will occur soon (*), and we need
> to think about how to work with OEMs on this. Currently most
> Samba OEMs are file serving only, and are on a combination
> of 3.5.x and 3.6.x.
> 
> I'm hoping that with the full release of Samba 4.0, we
> might be lucky and get a whole host of very welcome new OEMs
> selling products around AD Domain servers. I'm not worried
> about these OEMs :-). These people are well aware of what
> we do and don't do (or they won't have a very good product :-)
> and hopefully are already working closely with members of
> the Team.
> 
> The file serving OEMs I work with are more conservative.
> Normally they stay a release or about 1 year behind our
> current latest code, for stability reasons. It's hard to blame
> them, storage is an inherently conservative business, and
> they have to wait until the consensus is that a release is
> good enough to run multi-petabytes (or even exabytes) of
> storage on.
> 
> So, realistically, most Samba storage vendors are going
> to stick with 3.6.x for the next year or so. The question
> is, how to provide them with functional updates on that
> code until they feel comfortable enough to go with the
> version of 4.0.x that is current ?

As Kai has already pointed out, this is not a new problem:
We did not have such a discussion when we moved from 3.5 to
3.6 and thereby froze 3.5, what makes this fundamentally
different in the move 3.6 -> 4.0 from the pure file server
perspective?

Only the number? Or the build system questions?
Or some of the architechtural changes under the hood?

And as has been pointed out by Andreas, entrerprise distributors
also have to keep the latest stable release active for a while
and hence suffer from a similar problem.

It have traditionally been the distributors and the storage
product vendors (aka OEMs) that have been doing the work
of backporting features and maintaining a set of backport
patches on top of a stable release. Partly with the help or
in personae of some of us, but not officially maintained
as a backport release by the samba team.

> I'm not talking about regular bugfixes here, but bugs like:
> 
> https://bugzilla.samba.org/show_bug.cgi?id=8926
> 
> "Please use vfs_shadow_copy2 from master".
> 
> Currently this is closed out, due to it being a functionality
> addition. I think we need to find a way to handle this
> without expecting all storage vendors to move immediately
> to the 4.0.x fileserver.

Well, again, this problem is not new and applies to any interesting
feature that is in the master/next version branch but not in currently
released code. OEMs and enterprise distributors have always been
stuck between a rock and a hard place: On the one hand side a lot
of the interesting development work in the file server area is
done triggered by these users of samba and made exactly for their
use cases, on the other hand side they can not afford to move
from the stable release to a new yet unstabilized release.

I do not see that this situation will be moved to a different level
with the impending release of 4.0. Jeremy, if you could make this
more apparent to me, that would really be useful.

> My feeling is we can do this with some version of carrot
> and stick. The carrot is we keep maintaining the 3.6.x
> codebase once 4.0.x is released - so they know that their
> current "stable" product remains supported and fixes come
> in a timely fashion. This is what we ordinarily commit to
> do for and are currently doing for 3.5.x.
> 
> The stick is that the SMB3.0 functionality which requires
> major structural changes to the core fileserver code is
> *only* done in 4.0.x and above.
> 
> However this leaves a grey area of functionality additions
> that are really useful (e.g. the aio_pthread code which
> I snuck into 3.6.x) but not large rewrites. So I can see two
> options here (there are more, but these are the ones I think
> are workable) :
> 
> 1). Once 4.0.x ships, relax the rules on functionality
> additions to 3.6.x so that we can add *some* limited new
> functionality like bugid 8926, or the pthread aio code,
> or the new VFS module someone wants to submit. *Maybe*
> VFS changes (not sure about that..).
> 
> We will of course commit to doing this only for a year or
> so. Requirements for new SMB3.0 functionality will attract
> OEMs to 4.0 eventually. This is *close* to what we're
> doing now, but it losens up the restrictions we currently
> have on adding new code.
> 
> 2). Once 4.0.x ships, fork off a 3.7.0 tree from the current
> 3.6.x tree and add the limited new functionality there. As
> this is post 4.0.x ship then we still recommend 4.0.x for
> all users, but we have the ability to add *limited* new
> functionality to a "trusted" release that is very close to
> what storage OEMs currently depend on.
> 
> What do people think ? My personal preference is for option
> (1), but I'm open to other ideas. The problem we have to
> solve is that OEMs do need some of these code changes, but
> will not immediately move to 4.0.x for a year or so. So
> whatever response has to at least address this problem - merely
> saying "let them use 4.0.x" isn't going to solve it, because
> trust me they won't.

For me, option (2) is completely unacceptable.
It is setting the wrong sign. We do want our OEMS to switch to
the 4.0 file server some time not too far in the future, don't we?
And I also don't think that we would have the resources to maintain
a second (even if minor) feature release in parallel to 4.0.X.

For option (1), I agree that we should/could enlarge the maintenance
window for the 3.6.X series of releases.

But I am not convinced that it would be a good idea to break our
rule not to backport features. Again, this eats resources.
It potentially destabilizes the release.
And the thing I am most afraid of is that we this grey area
of what may backported and what may not will have unclear
boundaries. People will try and stretch the rules and try to get
their desired patches in at the risk of destabilization.

So I would rather stick with Kai's poposed option (3)
to not open 3.6.X for feature backports but stick to our rule.


That does of course not mean that I don't want to support our OEMs.
Of course I do!
But I am not convinced that centralizing this by opening the
stable release for feature backports is the right way.

Instead, if we want to provied a centralized platform for
backported patches, we could maintain a collection of backported
patches of some form without a complete release machinery hanging
off of it. For example, we could create a website with lists of
patchsets that can be applied to the 3.6 releases.
If this is too cumbersome we could possibly create one a backport
branch for 3.6 that is ahead of current v3-6-stable by some
backported patchsets. Or even create multiple branches for
different focusses, a branch for each backport topic would at
least be thinkable. (This would be similar to the clustering
branches v3-6-ctdb.)

This would be a means for providing backported patches for
our OEMs and distributors (who are highly welcome to provide
their own backported patches to this platform) without the
overhead and the guarantees attached to a stable release.

Wouldn't that be a good compromise?

Cheers - Michael

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 206 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20120606/9f416c9c/attachment.pgp>