[PATCH] Python 3 porting for selftest.ndrdump

[Noel Power]

Hi Joe,

First of all great to see another body working on this. However I really
fear that unless we are careful there is a danger of duplication of
effort here with some work that has already been done by Lumir, myself &
David. Actually the patch you are working on falls into this category
(in the patch set mentioned in point 2 below)

Regarding your plan below this is very similar to what we have tried. At
suse myself and David found quite a number of c-modules not converted to
python3. Lots of tests import modules which import other modules etc.
etc. as a result there can be quite a complicated mess of
interdependencies. Of course there are probably quite a fair number of
tests that can be enabled that don't require these missing 'c-modules'
and perhaps our experience was coloured by the tests that we tried to
enable. However it seemed to be successful and avoid any potential
import problems a good base to start from would be the availability of
all python3 modules so our plan is to

 1. Port all the remaining c-modules that are used to need be converted
to python3. We think we have all of those converted. We have some of
those c-modules out for review.

    https://github.com/samba-team/samba/pull/147
    https://github.com/samba-team/samba/pull/125
    https://github.com/samba-team/samba/pull/126
    https://github.com/samba-team/samba/pull/127

Unfortunately there has been little progress with the above and
currently they are all afaics stalled.
Additionally we have quite a few more modules ported (but not for review
yet) some we are trying to write tests for,  and for others the existing
tests depend on patches like the above and are also stalled because of that.

 2. There is a range of  python2 -> python3 known porting issues (such
as exception declarations,  number format, calls to print methods etc.)
Where we have experienced some of these and where it seemed like a bulk
change could be possibly with little risk we have tried to that. There
is still an outstanding review for the last one submitted

   https://github.com/samba-team/samba/pull/146

   There probably will be more or these (things like iteration behaviour
come to mind) these might be more complicated to change en-masse.

3. Start enabling piecemeal tests (of course we currently are focussing
on tests that are impacted/test the modules we are porting) The
intention though is to extend this once complete.


I am not sure what is the best way to proceed is to avoid duplication
and stepping over each others toes. Personally I don't think it makes
sense to block everybody else's progress with some long living branch
the everybody depends on. I would prefer if we could commit early and often.

>From my point of view the best way to get this working is to get the
outstanding reviews (and the queued up c-modules not currently out for
review) into master as soon as possible. I do believe that getting the
code in sooner rather than later is the best way forward as long as we
are as sure as we can that it doesn't impact the current python2

The problem with the python3 port is not trivial, there needs to be a
critical mass of changes in place in order to be able to start porting
existing tests effectively. Not being domain experts with the many many
areas the code touches we will have to rely on getting tests enabled to
actually see potential problems with the api. The sooner the code is in
the sooner we will see those problems. Hopefully the changes we make
will only affect python3 behaviour so we have a window of opportunity to
shake down python3 whilst maintaining python2 working as normal.

Noel


On 21/03/18 01:41, joeg at catalyst.net.nz wrote:
>
> Hi everyone:
>
> I am going to focus on the Python 3 porting task for 2 weeks or so.
>
> I was a Python Web developer before.  In this context, most of my
> knowledge are about Python,
>
> and I don't know too much about Samba internal yet.
>
> To make this Python 3 porting task measurable and operational to me,
> after a discussion with
>
> Andrew Bartlett, I decided to carry on it via selftest:
>
>
> 0.  Configure samba with `--extra-python /path/to/python3`
>
> 1.  Find a test suite in selftest/tests.py,  add `py3_compatible=True`.
>
> 2. Run above test suite, and fix errors.
>
> 3. Send patch to email list.
>
> 4. Move on to next one.
>
>
> This is the first patch, for `samba.tests.blackbox.ndrdump`, and it's
> simple. 
>
> In the meanwhile, I will work on next ones and send follow-up patches.
>
> Let's see whether this strategy works well.
>
> Review appreciated. Thanks!
>
>
> Joe Guo
>
> -- 
> Joe Guo
> joeg at catalyst.net.nz
> Catalyst IT