Converting SMB1 tests to SMB2

[Noel Power]

Hi All

Just thought I summarize what we (myself & Ralph) discussed, for my own
benefit and the benefit of others.

We agreed that

   * getting something upstream that would allow us to iteratively port 
tests is the best start, long living branches, rebase horror and a final
dump of work at some unknown future time is not ideal.

  * we need to keep the smb1 tests running until such time as the smb1
code is removed

  * we need some both a way to easily identify what needs to be ported
(or investigated [1]) and then mark them as done so anyone who want to
help or contribute knows what has already been ported and addressed or not

To achieve the above the plan is to

a) switch environments to by default be SMB2 only (e.g. set the min
protocol to SMB2_02)

b) identify the tests that currently fail via a new skip file [2] and
use that skip file as part of the normal test run thus showing with a
successful CI run that all remaining tests run with the now 'default'
SMB2 test environments, all failing tests don't run

c) make the failing tests run once more by moving the tests (identified
in the skip file) to new test environments that have the 'old' min
protocol options set add these test environments to the existing
autobuild jobs. E.g. the 'samba-ad-dc-5' autobuild job uses the test
environment 'ad_dc_default', so in this case we would create a new test
environment 'ad_dc_default_smb1' and we would adjust the 'samba-ad-dc-5'
autobuild job to use 'ad_dc_default' and additionally
'ad_dc_default_smb1' test environments. Same then for each autobuild job

d)  don't use the new skip file in the test run anymore, it's just for
documentation

With such changes in place upstream we can then go through the list of
tests and process them. By processing we mean that the test is either

i) left alone, this test is testing functionality that is SMB1 only
(example testing specific SMB1 messages) so we just mark this as done.
We could mark this with a comment perhaps in the new skip file. We also
intent to create a trello page to track this too

ii) the test is ported, the new smb2 test needs to be added to the non
'_smb1' environment, the existing test needs to remain in place for now.
E.g. say for example we have a test 'samba3.foo'.bar' that runs against 
'ad_dc_smb1' then after porting we need to create a new test entry in
'source3/selftest/tests.py'  for  'samba3.foo'.bar' to run against 'ad_dc'

I have been trying to get a branch together to achieve the above that
would provide the framework to allow us to start porting tests, the
current WIP branch is
https://gitlab.com/samba-team/devel/samba/commits/npower_exclude_smb1failures_squash

There have been some issues, firstly it has been very time consuming and
tedious to get to the stage where the initial skip file was created and
then again to the point where all the involved tests are moved to the
new 'xyz_smb1' test environments. And after all that some more issues :/


problem 1)

The mixture of xyz & xyz_smb1 test environments seems to cause strange
issues (see
https://gitlab.com/samba-team/devel/samba/pipelines/92945128), many
tests fail or hang. Running the tests against these environments
individually works no problems. Mostly I guess this is an issue with the
fact that the NETBIOS name of these 'duplicated' environments are the
same and this can cause clients to connect to the wrong server (from the
wrong environment)

problem 2)

Yes, not being a test environment guru and being maybe a little lazy and
I wanted to avoid trying change the test environment foo to change or
pass down NETBIOSNAME  (seemed a little risky too, maybe there is other
things like this).  So I decided to create 2 new autobuild jobs to run
the smb1 tests. Unfortunately these fail in CI (see
https://gitlab.com/samba-team/devel/samba/pipelines/93029325) What is
really weird here is

    a) running these autobuild jobs locally on my own machine causes no
problems, both jobs pass

    b) running these jobs in the *same* docker container as CI uses
(using the same autobuild command) again works no problems

    c) running the problematic jobs on sn-devel-184 using
autobuild-private-security.sh again works :-(

I am trying (sofar unsuccessfully) to figure out why these are failing

Noel

[1] it looks like some tests that are expected to pass against
environments that are SMB2 only don't.. It maybe these tests aren't
really SMB2 or perhaps use internally some SMB1 messages or something
else is afoot

[2] Using knownfailures was also an option but unfortunately some of the
tests that fail in the SMB2 only environments fail by crashing or
hanging so the easiest solution was just to skip these tests

On 23/10/2019 13:26, Ralph Boehme wrote:
> On 10/23/19 2:02 PM, Noel Power wrote:
>> On 23/10/2019 12:57, Ralph Boehme via samba-technical wrote:
>>> On 10/23/19 1:08 PM, Ralph Boehme via samba-technical wrote:
>>>> On 10/23/19 12:38 PM, npower wrote:
>>>>> * I thought that it would not be acceptable to just effectively remove all of these tests without a clean transition path, e.g. they run till they are removed
>>>> All this has to happen in a private branch of course, you can't push the
>>>> initial changes upstream of course.
>>> what would works as well, and I guess this might be what you had in mind, is
>>>
>>> * force smb2 in the testenvs
>>>
>>> * move all failing tests to seperate temporary testenvs that still allow
>>> smb1
>>>
>>> This changeset could the already go upstream, allowing working on the
>>> indivual tests in a more piecemeal fashion, because the resulting work
>>> can be pushed upstream for every test.
>>>
>>> Once all tests are taken care of, the temporary test envs can be removed.
>>>
>>> This would avoid accumulating all the changes in a private branch and
>>> the rebase hassle that comes with it.
>>>
>>> Is this what you had in mind?
>> exactly, because of the confusion I probably didn't articulate what I
>> was thinking clearly enough, sorry about that :-))
> ok, glad we figured that out. :)))
>
> My phone number is in the team repo, feel free to call me anytime or
> ping me on irc.
>
> -slow
>