[PATCH] Python 3 porting for selftest.ndrdump

[joeg at catalyst.net.nz]

Hi Noel,

Thank you for the great input, it's very helpful to me.

I agree with your plan, we need to port the c-modules first, and then
python.

I will get all the pending python3 porting related patches together, and
trigger autobuild with gitlab ci.

If the result is good, then we are more confident to get the code in,
and move on to next issue.

Let's see what will happen.


On 22/03/18 06:14, Noel Power via samba-technical wrote:
> Hi Joe,
>
> First of all great to see another body working on this. However I really
> fear that unless we are careful there is a danger of duplication of
> effort here with some work that has already been done by Lumir, myself &
> David. Actually the patch you are working on falls into this category
> (in the patch set mentioned in point 2 below)
>
> Regarding your plan below this is very similar to what we have tried. At
> suse myself and David found quite a number of c-modules not converted to
> python3. Lots of tests import modules which import other modules etc.
> etc. as a result there can be quite a complicated mess of
> interdependencies. Of course there are probably quite a fair number of
> tests that can be enabled that don't require these missing 'c-modules'
> and perhaps our experience was coloured by the tests that we tried to
> enable. However it seemed to be successful and avoid any potential
> import problems a good base to start from would be the availability of
> all python3 modules so our plan is to
>
>  1. Port all the remaining c-modules that are used to need be converted
> to python3. We think we have all of those converted. We have some of
> those c-modules out for review.
>
>     https://github.com/samba-team/samba/pull/147
>     https://github.com/samba-team/samba/pull/125
>     https://github.com/samba-team/samba/pull/126
>     https://github.com/samba-team/samba/pull/127
>
> Unfortunately there has been little progress with the above and
> currently they are all afaics stalled.
> Additionally we have quite a few more modules ported (but not for review
> yet) some we are trying to write tests for,  and for others the existing
> tests depend on patches like the above and are also stalled because of that.
>
>  2. There is a range of  python2 -> python3 known porting issues (such
> as exception declarations,  number format, calls to print methods etc.)
> Where we have experienced some of these and where it seemed like a bulk
> change could be possibly with little risk we have tried to that. There
> is still an outstanding review for the last one submitted
>
>    https://github.com/samba-team/samba/pull/146
>
>    There probably will be more or these (things like iteration behaviour
> come to mind) these might be more complicated to change en-masse.
>
> 3. Start enabling piecemeal tests (of course we currently are focussing
> on tests that are impacted/test the modules we are porting) The
> intention though is to extend this once complete.
>
>
> I am not sure what is the best way to proceed is to avoid duplication
> and stepping over each others toes. Personally I don't think it makes
> sense to block everybody else's progress with some long living branch
> the everybody depends on. I would prefer if we could commit early and often.
>
> From my point of view the best way to get this working is to get the
> outstanding reviews (and the queued up c-modules not currently out for
> review) into master as soon as possible. I do believe that getting the
> code in sooner rather than later is the best way forward as long as we
> are as sure as we can that it doesn't impact the current python2
>
> The problem with the python3 port is not trivial, there needs to be a
> critical mass of changes in place in order to be able to start porting
> existing tests effectively. Not being domain experts with the many many
> areas the code touches we will have to rely on getting tests enabled to
> actually see potential problems with the api. The sooner the code is in
> the sooner we will see those problems. Hopefully the changes we make
> will only affect python3 behaviour so we have a window of opportunity to
> shake down python3 whilst maintaining python2 working as normal.
>
> Noel
>
>
> On 21/03/18 01:41, joeg at catalyst.net.nz wrote:
>> Hi everyone:
>>
>> I am going to focus on the Python 3 porting task for 2 weeks or so.
>>
>> I was a Python Web developer before.  In this context, most of my
>> knowledge are about Python,
>>
>> and I don't know too much about Samba internal yet.
>>
>> To make this Python 3 porting task measurable and operational to me,
>> after a discussion with
>>
>> Andrew Bartlett, I decided to carry on it via selftest:
>>
>>
>> 0.  Configure samba with `--extra-python /path/to/python3`
>>
>> 1.  Find a test suite in selftest/tests.py,  add `py3_compatible=True`.
>>
>> 2. Run above test suite, and fix errors.
>>
>> 3. Send patch to email list.
>>
>> 4. Move on to next one.
>>
>>
>> This is the first patch, for `samba.tests.blackbox.ndrdump`, and it's
>> simple. 
>>
>> In the meanwhile, I will work on next ones and send follow-up patches.
>>
>> Let's see whether this strategy works well.
>>
>> Review appreciated. Thanks!
>>
>>
>> Joe Guo
>>
>> -- 
>> Joe Guo
>> joeg at catalyst.net.nz
>> Catalyst IT
>

-- 
Joe Guo
joeg at catalyst.net.nz
Catalyst IT