Fixing sn-devel only (was: Re: bashism again (was: Re: LDAP tests failing))

[Andrew Bartlett]

On Sat, 2018-02-10 at 09:41 +0100, Andreas Schneider wrote:
> On Friday, 9 February 2018 19:59:32 CET Andrew Bartlett via samba-technical 
> wrote:
> > On Fri, 2018-02-09 at 19:14 +0100, Ralph Böhme wrote:
> > > can we just replace dash with bash on sn-devel?
> > > 
> > > Given the grieve and headache this is causing and the staggering amount of
> > > work adding test for the shell scripts itself, paving over the problem by
> > > replacing dash with bash looks tempting.
> > 
> > I'm sorry, while incredibly tempting, just 'fixing' sn-devel would just
> > make the problem worse.
> > 
> > The reason is this.  While the vast majority of Samba developers only
> > ever test Samba on sn-devel as part of running an autobuild, it is not
> > the only test platform in use.
> > 
> > * For my colleagues as Catalyst, we also test Samba on VM images of
> > Ubuntu 14.04 on the Catalyst Cloud.
> > 
> > * An environment (based on a Docker image) of Ubuntu 14.04 is being
> > prepared to test Samba via Gitlab CI
> > 
> > * Travis CI tests Samba for contributors that use GitHub, running on
> > Ubuntu 14.04
> 
> Do you see the problem here?

No, but depending on how we fixed the problem (an inverted test) and
fixed sn-devel to run bash as then it would fail there. 

> All test machines are Ubuntu. Till now I'm the only one who runs 'make test' 
> on his developer boxes (Fedora and openSUSE) to find and fix issues.
> 
> If everything is Ubuntu we keep not finding bugs because everything is just 
> Ubuntu ...

Sure, but I don't want to lock in that failure.  

> > * Despite the 5 hour runtime, some developers run the full make test
> > locally on their development workstations and many developers run the
> > test they think they are fixing locally.
> 
> Probaby not or only on Ubuntu.

It has got a bit over-whelming, I agree.

If my vision for cloud CI in Gitlab for Samba ever gets anywhere,
running a a full set or subset of tests on different docker images
(distributions) should be quite practical.

Thanks,

Andrew Bartlett

-- 
Andrew Bartlett                       http://samba.org/~abartlet/
Authentication Developer, Samba Team  http://samba.org
Samba Developer, Catalyst IT          http://catalyst.net.nz/services/samba