autobuild requires lmdb (was: Re: WAF 2.x upgrade status)

[Martin Schwenke]

I may already have a fix for the event script test failure. I was seeing 
similar issues the other day due to an uninitialised test variable.

Tomorrow...

peace & happiness,
martin

On 4 July 2018 9:58:21 pm Alexander Bokovoy via samba-technical 
<samba-technical at lists.samba.org> wrote:

> On ke, 04 heinä 2018, Andrew Bartlett via samba-technical wrote:
>> On Tue, 2018-07-03 at 23:06 +0300, Alexander Bokovoy wrote:
>>> On ke, 04 heinä 2018, Andrew Bartlett via samba-technical wrote:
>>>> On Tue, 2018-07-03 at 14:10 +0300, Alexander Bokovoy wrote:
>>>>> - it is not possible to run full autobuild on a system with ldb without
>>>>> lmdb. This is reproducible on Gitlab runners as well.
>>>>
>>>> This is by design.  I didn't even what such builds to be possible for
>>>> this reason, but we got caught by the 32bit implications at the last
>>>> moment.
>>>
>>> That's not my point. Why other builds are successfull then with the same
>>> runners in gitlab if they are supposed to fail?
>>
>> So, to be more specific (thanks for the CI link, I should have followed
>> it before, and it demonstrates how valuable this is:
>>
>> The "test" job in the "ldb" autobuild target in the "samba-others" CI
>> job as demonstrated here fails:
>>
>> https://gitlab.com/samba-team/devel/samba/-/jobs/79013256
>>
>> It fails with "make test called, but ldb was built --without-ldb-lmdb"
>> however that is not true (the --without-ldb-lmdb stage is next, called
>> "configure-no-ldb").
>>
>> We know this because of the configure output:
>>
>> Checking for lmdb >= 0.9.16                                                 
>>                     : not found
>> Checking for header lmdb.h                                                  
>>                     : yes
>> Checking for lmdb >= 0.9.16 via header check                                
>>                     : ok
>> Checking for library lmdb                                                   
>>                     : yes
>> Checking for mdb_env_create                                                 
>>                     : ok
>>
>> That suggests that while the mechanism used to pass the status as to if
>> LMDB was built is no longer connected in the same way between the
>> *configure* and *make test* components of waf.
>
> Found it -- samba_utils.LOAD_ENVIRONMENT() assumed we run at top level
> in Thomas Nagy's patches to port Samba to waf 1.9. Fixed and now it
> works.
>
> A ctdb issue turned out to uncover few more unported pieces around
> supporting different compilers -- mostly because waf changed from using
> 'cc' to 'c' in internal terminology and older python imports became
> wrong.
>
> ctdb had also been doing a weird dance around generating VERSION value
> in ctdb/wscript which broke completely for scope/context isolation in
> newer waf (since 1.8, I think). Took some time to figure that out.
>
> I'm not done with ctdb yet because my changes uncovered some of bugs in
> ctdb tests, mostly around assumptions that certain binaries are
> available in path during the test. Interesting that in Gitlab runners a
> failing test is tests/onnode/0001.sh which is not able to find 'onnode'
> command while in local autobuild runs I'm getting failure in
> tests/eventscripts/10.interface.multi.001.sh
>
> --==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--
> Running test 
> /tmp/samba-testbase/b29812/prefix/ctdb/share/ctdb/tests/eventscripts/10.interface.multi.001.sh 
> (14:42:03)
> --==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--
> 10.interface      multi      001  - takeip, removeip
>
> Setting up PNN 0
> Setting up public addresses in 
> /tmp/tmp.tEC7tVEfOQ/unit_eventscripts/etc-ctdb/public_addresses
> Setting up CTDB with 3 fake nodes
> ==================================================
> Running script "10.interface.script takeip"
>
> ##################################################
> CTDB_BASE="/tmp/tmp.tEC7tVEfOQ/unit_eventscripts/etc-ctdb"
> CTDB_SYS_ETCDIR="/tmp/tmp.tEC7tVEfOQ/unit_eventscripts/etc"
> ctdb client is 
> "/tmp/samba-testbase/b29812/prefix/ctdb/share/ctdb/tests/eventscripts/stubs/ctdb"
> ip command is 
> "/tmp/samba-testbase/b29812/prefix/ctdb/share/ctdb/tests/eventscripts/stubs/ip"
> --------------------------------------------------
> Output (Exit status: 1):
> --------------------------------------------------
> ERROR: must supply interface, IP and maskbits
> --------------------------------------------------
> Required output (Exit status: 0):
> --------------------------------------------------
>
>
> FAILED
> ==========================================================================
> TEST FAILED: 
> /tmp/samba-testbase/b29812/prefix/ctdb/share/ctdb/tests/eventscripts/10.interface.multi.001.sh 
> (status 1) (duration: 1s)
> ==========================================================================
>
>
> The very same test passes in Gitlab runner.
>
> --
> / Alexander Bokovoy