[PATCH] Solve issues with flapping patches

[Andrew Bartlett]

On Fri, 2017-06-30 at 14:28 +1000, Martin Schwenke via samba-technical
wrote:
> On Fri, 30 Jun 2017 16:20:06 +1200, Andrew Bartlett
> <abartlet at samba.org> wrote:
> 
> > On Fri, 2017-06-30 at 14:09 +1000, Martin Schwenke via samba-
> > technical
> > wrote:
> > > Sorry if I'm wrong here.  I can happily live with occasional
> > > autobuild
> > > failures... but I find the constant failures very
> > > frustrating...   
> > 
> > That would leave the most critical components of the AD DC
> > untested,
> > which isn't acceptable either.
> 
> As I said, the tests can be switched back on by the developers who
> want
> to test that code.  So, all of your testing can run those tests.
> 
> The autobuild process is used as a gatekeeper for getting good
> code/tests into the tree.  It doesn't help you when I test critical
> components of the AC DC and flakiness causes my autobuilds to
> fail! You don't even notice it... but I do...  :-)

Actually, it does.  See, the thing about shared development is that we
have found, despite all our best hopes, that many changes have
unintended consequences in Samba.  

Regularly patches that are tested against the 'things they should
impact' fail the full autobuild, not because of flapping tests, but
because the developer didn't foresee that what they would break. 

Having a large, some might say even moderately comprehensive testsuite
is one of the few reasons we even have an AD DC, because had we not
moved to autobuild before the time the AD resources collapsed, we could
never have released Samba 4.0.  Instead, we would still be chasing our
tails trying to undo the unintentional regressions. 

That is why, despite the emotional satisfaction, sweeping un-targeted
entries in flapping do more harm than good.

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett
https://samba.org/~abartlet/
Authentication Developer, Samba Team         https://samba.org
Samba Development and Support, Catalyst IT   
https://catalyst.net.nz/services/samba