How to move storage OEMs to Samba 4.0 ?
Michael Adam
obnox at samba.org
Tue Jun 12 00:21:06 MDT 2012
OK, my mail was way too long.
Let me recap the most important questions/points:
* What is so new in the 4.0 release step file-server-wise
that OEMs will refrain from upgrading even longer than
they did/do for other releases?
Is 4.0 fileserver-wise really worse / more scary / bigger than
earlier releases from the point of view of OEMs?
* The major changes listed to support the dangerousness of 4.0 were
changes not authored by you (or Volker, or me..). So might it
be that this fact rather creates an increased fear (of the unknown
or uncontrolled) on your side?
* In short, is it really the OEMs' fears your are expressing or
rather your own fear projected to the OEMs?
Of course I want to help the OEMs (I already do).
But I am not convinced that that opening up 3.6 for feature
backports is the right thing.
* It would be the wrong sign (to not move to 4.0).
* Opening up patch policy for the release increases the risk
of destabilization, even if (of course) not intended.
* I would agree to allow backports of isloated new code like vfs
modules or command line tools where risk is minimal.
* What do the OEMs themselves want from us?
Do they want a blended OEM-feature release?
* We could offer a loose collection of backported feature patches
not harnessed into a release, so every interested OEM can pick
from that collection.
I would really like to discuss this further since I also want to
reach the best possible way to help OEMs with current releases
while encouraging them to move to 4.0.
Cheers - Michael
Michael Adam wrote:
> I apopologize for the length of the mail up front.
> But I have been sitting all night over it, elaborating some
> thoughts...
>
> Jeremy Allison wrote:
> > On Wed, Jun 06, 2012 at 11:49:05AM +0200, Volker Lendecke wrote:
> > >
> > > It is the sum of changes that went in. gensec,
> > > security=share gone, security=ads requires a more advanced
> > > environment, merged loadparm, possibly ntdb as a new
> > > default. This is enough to scare people away from s4 for the
> > > OEM case for at least 12 months if not much more from now.
> > > I would say that we can't leave those in the cold for so
> > > long. We have a backlog of features that we don't accept
> > > right now, and this will only grow over the next year or
> > > two.
> >
> > This is the truth of it. Seriously. Volker and I have
> > a lot of experience working with OEMs and you really
> > need to trust us on this.
>
> Well. I don't see what is new here. What is so different
> in this release step compared to the previous release
> upgrades?
>
>
> Let me try an analyis:
>
> The situation for the OEMs that you have described seems
> to be pretty much the same for this release as for previous ones:
>
> * OEMs love our stable releases for their stability.
> * OEMs yearn for using the new release for the features.
> * But OEMs fear the new release for its potentially missing stability.
> * Therefore OEMs stick to stable release for >= 1 year
> after new release has been made and backport some of
> the features to the stable release they use.
>
> It has been like this before, for many releases.
> And it will be like this for 4.0.
>
> But now it is suddenly a more severe probelm?
>
> Is it really that OEMs themselves have an especially increased
> fear of the 4.0 release? Maybe because of the major version bump?
> Or because of some technical insight in what has gone one
> in source3 since 3.6?
>
> Or is it rather that you yourself are more afraid of the next
> release 4.0 than you have been of previous releases?
> The things that Volker has listed above as changes
> are all changes that none of you (or me) have authored.
> That would be not so bad an explanation of increased fear. :-)
>
> But this would not be a fear that I'd expect the OEMs to share.
>
> For users (like the OEMs) that have a main interest in the smbd
> file server, this release will appear different most visibly
> in that there is a new mode of building the server. Then there
> will be (if we manage to get it in in time) a whole new set of
> SMB2 features (durable handles (of 2.0), dynamic reauthentication
> (of 2.1) and the basic 3.0 support) - not in yet. Finally there
> are many smaller or bigger changes, performance
> improvements, vfs modules and whatnot. (If I missed something
> here, I apologize, and please fill it up.) A couple of legacy
> features are about to be dropped. -- So I am convinced that from
> this perspective, the release must look very similar to earlier
> releases.
>
>
> Detour: view from inside samba
>
> But from inside samba, for those who have worked on the samba3
> file server with complete dedication for many years, and
> virtually had the code under their control, those not so publicly
> visible changes that are a byproduct of the merge-samba3-and-samba4
> efforts, may be able to create more fear than any ever so big
> changes they have done themselves. This is quite understandable.
> And apart from all the useful things that the code-merge (imho)
> creates, there is a real danger lurking:
> This danger is is called nested event loops.
>
> I hope to quote Volker correctly in trying to summarize it
> briefly: calling an event loop from within an event handler on
> the same event context, the world has changed for the event
> handler when the inner event loop returns. This can lead to
> arbitrary behaviours from infinite nesting (i.e. stack
> exhaustion) to unexpected socket state and so on. This pattern is
> therefore banned in samba3 code. In samba4 code, the pattern is
> used in several central places. (Maybe the chances to hit this
> problem are higher in the central event loop of a file server
> that is under real pressure than in the AD server, at least the
> samba3-devs have been more strict about this point.)
>
> The use of nested event loops in samba4 is a design that some agree
> should be changed. But it will not happen very soon because it
> will be non-trivial to do and require a lot of effort. So when the
> merge project creates subsystems in the base directory these
> subsystems often contain the potential to create nested event
> loops, when they originate from source4/. By using new merged
> subsystems in source3/, the possibility is created to get nested
> eventloops in through the back door. S3 code is currently
> protected against it in that it panics when a nested event loop
> is called. And there are a few more or less implicit mechanisms
> to control the behaviour of the base subsystems in question.
> But this is fragile. If someone misuses it, it will still panic
> but it would be best to instead have an obvious and foolprof
> protection mechanism that would forbid s3 to use the base
> libraries in a way to allow for nested event loops.
>
> I think that this is the biggest internal concern about the
> state of the smbd file server in the 4.0 release.
> Summing up, I think the state of the file server is in fact good!
> In order to keep it safe we might think about better
> protection of the misuse of base libraries in source3/ .
>
>
> Back to OEMs:
>
> This was a detour about the internal perception of the file
> server in the 4.0 release. Coming back to the perception of
> our OEMs and what we should tell them and how we can help them.
>
> As (my thesis is that) the state of the file server in 4.0 is
> generally quite good, I think we should encourage the users
> (OEMs/distributors) to use the file server from the new release
> for new features in the same way that they have done in previous
> releases. Fileserver-wise this will be a normal release. Not a
> small one, but also not a complete overhaul.
>
> The OEMs have created and maintained backport patchsets for their
> product in the past. And they will do so for 3.6 when 4.0 is out.
> The question raised is how to best support them in doing this.
>
> I doubt that maintaining a release that contains backported
> features from various OEM patchsets will be the right way to do
> this. Kai has nailed some downsides of this approach in his
> several mails. Let me repeat some and add some thoughts:
>
> * Each OEM maintains his own set of patches that the OEM itself
> tests carefully.
>
> If we blend subsets of the various OEMs' patchsets into a
> release, it is by no means certain that the result will
> be useful or stable for all of them, because each OEM has
> only QA'd his own patchset, not the blend.
>
> More importantly, it is uncertain that this release will
> be what most OEMs want! (feature-wise)
>
> Put more pointedly: We also don't have a round table of OEMs
> that discusses the common denominator of all OEMs' patchsets.
>
> I consent that it can be expected though that many of the OEMs
> woud switch to such a feature-backported version if the patches
> had been choosen carefully.
>
> * Some or most(?) of the OEMs also have patches that they would not
> want to go upstream. So we wont't be able to take the need
> to maintain a patcheset on top of the release from the OEMs
> completely. The patchset just might get a little smaller.
>
> * Doing more of such backports for a release will create
> additional load. If you really want to do this as a service
> for the OEMs we should probably do it for 3.5.X as well,
> since this is what to my knowledge most of them are still
> using. So even more load. And we are already short of
> development ressources.
>
> * I can also imagine that adding features to the 3.6.X releases
> (and not to earlier releases, say 3.5.Y) right before 4.0
> comes out, will spread FUD about the 4.0 release and
> will deter users from switching rather than encouraging them.
>
> * The rule to not add features to the stable release branch
> is there to protect the stability of the release and to
> protect our resources, so that we can concentrate more
> on the newer release.
>
> Of course, backporting features does not necessarily
> mean that the branch gets utterly destabilized, but frequently
> a new feature will introduce new bugs that need subsequent
> bugfixes to be backported as well, and so on. I.e., by
> introducing new bigger hunks of new or massively changed
> code, we usually also introduce new bugs. This is some
> form of destabilization.
>
> * Also the notion of what is a reasonable and useful feature
> to backport is completely arbitrary. In saying you want to
> "relax the rules on functionality additions to 3.6.x so that we
> can add *some* limited new functionality", what do you imagine
> more concretely? This sounds so vague that it will certainly
> lead to dissensus and create room for sneaking stuff in. Where
> to draw the line? This is difficult. The current rule is
> simple (but cruel). :-)
>
>
> To sum up:
>
> * I don't think the OEMs will perceive the 4.0 release as
> especially theateneing -- file-server wise.
> So why is it more important now to backport features into
> the stable release than it has been before?
>
> * Creating a release with backports will not remove the need
> to maintain individual patchsets from our OEMS completely.
>
> * It may not even be what they want.
> What *do* the OEMs themselves want from us?
>
> * Opening up patch policy for the release does create the
> danger of destabilization. Of course this is not intended
> but the danger is there.
>
> * I could agree to allow backports of isloated new code like vfs
> modules or command line tools.
>
> * Can we start out with offering a loose collecting of useful
> backport patches from OEMs and form elsewhere in a different
> place than a release? It could be bugzilla or a webpage or
> whatever. No guarantees no release process...
>
> I don't have my final conclusion yet.
>
>
> Good night - Michael
>
--
Michael Adam <ma at sernet.de>
SerNet GmbH, Bahnhofsallee 1b, 37081 Göttingen
phone: +49-551-370000-0, fax: +49-551-370000-9
AG Göttingen, HRB 2816, GF: Dr. Johannes Loxen
http://www.sernet.de, mailto:kontakt at sernet.de
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 206 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20120612/a9bf44a4/attachment.pgp>
More information about the samba-technical
mailing list