[PATCHES] Build pytalloc for two Python versions at once, port to py3

[Petr Viktorin]

On 02/26/2015 11:36 PM, Andrew Bartlett wrote:
> On Thu, 2015-02-26 at 12:23 +0100, Petr Viktorin wrote:
>> On 02/25/2015 07:53 PM, Andrew Bartlett wrote:
>
>>> As examples, I'm particularly concerned with what the interface for and
>>> user users of our ldb and ndr generated code will look like.
>>>
>>> In particular, in ldb, we transport binary data and string data in the
>>> same ldb_val structures, and other than by convention and schema, there
>>> is no indication as to what one might find, yet adding binary2unicode
>>> conversions all over all our scripts would be a total nightmare.
>>
>> Hello,
>> I understand the concern, the unicode/bytes split will undoubtedly be
>> the hardest part of the port.
>> But that's part of the reason why I chose talloc, which doesn't deal
>> with strings, to get the buildsystem changes down. IOW, this is not
>> about the ldb/ndr interfaces.
>>
>> My assumption here is that Samba will need to port eventually, so there
>> will be some time when we need to build for both 2 and 3 at the same
>> time, at least for some parts. I believe that is true regardless of how
>> (for example) the bytes/unicode split is handled. Do you have a
>> different opinion here?
>
> I would really like to see how that can be handled.  The issue is, if
> that can't be handled well, then we will have all this porting
> infrusturcutre, and an expectation that we are open to a further move,
> without that actually being the case.  How the string handling works out
> really is a blocker for me on this, I would rather Samba be python2
> forever than get this wrong.
>
> We have the luxuary of time in our spot in the ecosystem, and while I'm
> very hesitant to turn down good work, I'm also really keen to continue
> to wait until the environment improves, as it seems to have slowly been
> doing.

What kind of environment improvements do you have in mind?
The last compatibility improvement in Python itself was in 3.4 (more 
than two years ago; the 'u' string prefix). The ecosystem of porting 
tools is still active, but the focus there is automating (or 
documenting) best practices and reducing boilerplate. (Also, these are 
tools for Python code: not much effort is put into tooling for C 
extensions.)

The unicode/bytes split is not going away. To use Python 3, Samba will 
need to decide what is a string and what is binary data. No environment 
improvements will make these decisions easier (though they might make 
working with the resulting code, or making the changes, easier).
Aside from this split, porting to Python 3 is, nowadays, quite 
mechanical. There are compatibility libraries like six and future, but 
they just provide is a convenience API. Even if we decide to use one, 
improvements to it will not make things significantly easier. Also, 
similar tools don't exist for C extensions, where a bunch of 
#ifdefs/#defines will do the job.
In short: the hard part (str/bytes split) is ineviable, and will not be 
helped much by environment improvements.

Another concern is that Python 2 will only be supported until 2020. If 
Samba doesn't port until then, chances are the Samba team will need to 
support Python 2.7 on its own.
There is time to do the porting, and I am willing to do it, but spending 
the time waiting for improvements to appear, or supporting only python2 
forever, doesn't strike me as a good idea.

>> I'd really like to have the buildsystem changes vetted before I tackle
>> the split, so I (and also the future reviewers) can focus on the porting
>> itself.
>
> That's understandable.  I'm hoping Jelmer might post some thoughts.  He
> and I came up with a reasonable set of expectations in some private
> mails going back and forth on this.

I'm quite interested in these thoughts. The last time we discussed this 
on the list, I got the impression that there was general agreement that 
the standalone libraries (like talloc) could be ported before the rest 
of Samba, supporting both versions of Python for a while.
If there are other expectations floating around in private, I would 
really like to hear them.


-- 
Petr Viktorin