failing tests in the testsuite on debian

[Ralph Boehme]

On 11/15/22 14:35, Michael Tokarev via samba-technical wrote:
> I experimented with the tests some more. Thank you Jeremy for the
> tip about escaping spaces in the knownfail files, - somehow I noticed
> this is the case in other places but didnt' think about using escapes
> in my "knownfails".
> 
> But the thing, or a request for help actually, is still here.  I found
> out that samba testsuite is unreliable.

sort of. It is just picky about choosing it's distro friends. :) A full 
pass can basically only be expected on the specific distribution that is 
used on gitlab CI and our final git gatekeeper (that runs a final make 
test before doing a git ff push). Iirc we're still using Ubuntu 18.04 on 
the git gatekeeper and the CI containers, cf:

<https://gitlab.com/samba-team/devel/samba/-/jobs/3324388385>

Another thing that might cause issues is that you seem to be running waf 
directly while you should use the make wrapper as that sets some 
important environment variables.

It seems your goal is running the full testsuite as part of packaging 
and I can only warn you that this likely a huge timesink. Eg some tests 
expect certain filesystems, maybe it's a more reasonable approach to 
only run a subset of tests that we run in our -o3 runners, eg:

https://gitlab.com/samba-team/devel/samba/-/jobs/3324388357

make test FAIL_IMMEDIATELY=1 TESTS='--exclude=selftest/slow-none 
--include-env=none'

or this one

https://gitlab.com/samba-team/devel/samba/-/jobs/3324388385

which includes more envs:

make testonly FAIL_IMMEDIATELY=1 TESTS='--include-env=fileserver 
--include-env=fileserver_smb1 --include-env=fileserver_smb1_done 
--include-env=maptoguest --include-env=ktest'

-slow

-- 
Ralph Boehme, Samba Team                 https://samba.org/
SerNet Samba Team Lead      https://sernet.de/en/team-samba

-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20221115/f06ea0ae/OpenPGP_signature.sig>