autobuild requires lmdb (was: Re: WAF 2.x upgrade status)

[Alexander Bokovoy]

On ke, 04 heinä 2018, Andrew Bartlett via samba-technical wrote:
> On Tue, 2018-07-03 at 23:06 +0300, Alexander Bokovoy wrote:
> > On ke, 04 heinä 2018, Andrew Bartlett via samba-technical wrote:
> > > On Tue, 2018-07-03 at 14:10 +0300, Alexander Bokovoy wrote:
> > > >  - it is not possible to run full autobuild on a system with ldb without
> > > >    lmdb. This is reproducible on Gitlab runners as well.
> > > 
> > > This is by design.  I didn't even what such builds to be possible for
> > > this reason, but we got caught by the 32bit implications at the last
> > > moment.
> > 
> > That's not my point. Why other builds are successfull then with the same
> > runners in gitlab if they are supposed to fail?
> 
> So, to be more specific (thanks for the CI link, I should have followed
> it before, and it demonstrates how valuable this is:
> 
> The "test" job in the "ldb" autobuild target in the "samba-others" CI
> job as demonstrated here fails:
> 
> https://gitlab.com/samba-team/devel/samba/-/jobs/79013256
> 
> It fails with "make test called, but ldb was built --without-ldb-lmdb"
> however that is not true (the --without-ldb-lmdb stage is next, called
> "configure-no-ldb").  
> 
> We know this because of the configure output:
> 
> Checking for lmdb >= 0.9.16                                                                     : not found 
> Checking for header lmdb.h                                                                      : yes 
> Checking for lmdb >= 0.9.16 via header check                                                    : ok 
> Checking for library lmdb                                                                       : yes 
> Checking for mdb_env_create                                                                     : ok 
> 
> That suggests that while the mechanism used to pass the status as to if
> LMDB was built is no longer connected in the same way between the
> *configure* and *make test* components of waf. 

Found it -- samba_utils.LOAD_ENVIRONMENT() assumed we run at top level
in Thomas Nagy's patches to port Samba to waf 1.9. Fixed and now it
works.

A ctdb issue turned out to uncover few more unported pieces around
supporting different compilers -- mostly because waf changed from using
'cc' to 'c' in internal terminology and older python imports became
wrong.

ctdb had also been doing a weird dance around generating VERSION value
in ctdb/wscript which broke completely for scope/context isolation in
newer waf (since 1.8, I think). Took some time to figure that out.

I'm not done with ctdb yet because my changes uncovered some of bugs in
ctdb tests, mostly around assumptions that certain binaries are
available in path during the test. Interesting that in Gitlab runners a
failing test is tests/onnode/0001.sh which is not able to find 'onnode'
command while in local autobuild runs I'm getting failure in 
tests/eventscripts/10.interface.multi.001.sh

--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--
Running test /tmp/samba-testbase/b29812/prefix/ctdb/share/ctdb/tests/eventscripts/10.interface.multi.001.sh (14:42:03)
--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--
10.interface      multi      001  - takeip, removeip

Setting up PNN 0
Setting up public addresses in /tmp/tmp.tEC7tVEfOQ/unit_eventscripts/etc-ctdb/public_addresses
Setting up CTDB with 3 fake nodes
==================================================
Running script "10.interface.script takeip"

##################################################
CTDB_BASE="/tmp/tmp.tEC7tVEfOQ/unit_eventscripts/etc-ctdb"
CTDB_SYS_ETCDIR="/tmp/tmp.tEC7tVEfOQ/unit_eventscripts/etc"
ctdb client is "/tmp/samba-testbase/b29812/prefix/ctdb/share/ctdb/tests/eventscripts/stubs/ctdb"
ip command is "/tmp/samba-testbase/b29812/prefix/ctdb/share/ctdb/tests/eventscripts/stubs/ip"
--------------------------------------------------
Output (Exit status: 1):
--------------------------------------------------
ERROR: must supply interface, IP and maskbits
--------------------------------------------------
Required output (Exit status: 0):
--------------------------------------------------


FAILED
==========================================================================
TEST FAILED: /tmp/samba-testbase/b29812/prefix/ctdb/share/ctdb/tests/eventscripts/10.interface.multi.001.sh (status 1) (duration: 1s)
==========================================================================


The very same test passes in Gitlab runner.

-- 
/ Alexander Bokovoy