[PATCHES] Build pytalloc for two Python versions at once, port to py3

[Petr Viktorin]

On 03/12/2015 03:41 PM, Jelmer Vernooij wrote:
> On Thu, Mar 12, 2015 at 03:13:26PM +0100, Petr Viktorin wrote:
>> On 03/12/2015 02:09 PM, Jelmer Vernooij wrote:
>>> On Thu, Mar 12, 2015 at 12:08:48PM +0100, Petr Viktorin wrote:
>>>> On 03/05/2015 10:08 PM, Andrew Bartlett wrote:
>>>>> On Wed, 2015-03-04 at 15:53 +0100, Jelmer Vernooij wrote:
>>>>>> On Wed, Mar 04, 2015 at 11:18:08AM +1300, Andrew Bartlett wrote:
>>>>>>> On Tue, 2015-03-03 at 22:09 +0100, Jelmer Vernooij wrote:
>>>>>>>> On Mon, Mar 02, 2015 at 12:09:25PM +0100, Petr Viktorin wrote:
>>>>> Petr,
>>>>>
>>>>> The reason I'm being so awkward is that I don't like any of the options.
>>>>>   - waiting forever means we:
>>>>>    - push aside your valuable contributions
>>>>>    - may never make the move
>>>>
>>>> I don't think the second point is true; you'd be forced to make the move
>>>> eventually, most likely when Python 2 support ends in 5 years. It's good not
>>>> to rush things, but waiting forever is hardly an option -- it would just
>>>> make us rush things later.
>>>
>>> If we move later, it saves us from a couple of years of juggling support
>>> for two different Python versions. In fact, if we move late enough, we might
>>> be able to just drop Python2 support altogether and just port to Python3 rather
>>> than supporting both versions.
>>
>> If there were no dependencies, there might be a chance of that working.
>> However, if you want to coordinate with FreeIPA or distros, it becomes
>> somewhat harder.
>> Also, allowing more time for the port would improve chances to do thigs
>> right, rather than rushing them all at once to avoid "juggling".
> We won't have to rush if we wait another year or two.
>
>> I liked Andrew's suggestion to support only Python 2, but start putting
>> unsupported py3 bits in "unofficially", so dependent projects can start
>> preparing their own py3 support before the big switch.
> Right, see https://lists.samba.org/archive/samba-technical/2015-March/105949.html for
> my thoughts on adding Python 3 support.

Sorry for the misattribution, it was your suggestion.

> The onus for adding support of Python3 is on those who care about it, and should
> not inconvenience the rest of us.

Alright, then we agree about this bit at least.
What in the patches proposed here would inconvenience you? Note that 
with these patches I *am* looking specifically at the stand-alone libraries.

>>>>>   - waiting for things to improve:
>>>>>    - you suggest is foolish, there is no indication that the python community will
>>>>>    - and we don't know that someone else will be keen to do the work in the future
>>>>
>>>> Also, there's a non-trivial chance that my work on Samba could drive these
>>>> improvements.
>>>> It's been suggested a CPython list that I write a porting guide for C code.
>>>> I'm quite tempted to share what I write for Samba.
>>>
>>> Thanks! Upstream improvements to make it easier to support both versions
>>> would be great.
>> Just note I wrote *porting guide*, not upstream code.
> Ah :-(
>
>>> I think defines to make it easier to compile 2.6 C code with Python3 belong
>>> in an upstream header (perhaps hidden behind and #ifdef _COMPAT_PYTHON26 or
>>> with a deprecation warning), rather than in every Python projects' source
>>> tree.
>> I don't think that is going to happen in Python itself. I can however make a
>> library in the spirit of six/future for C extensions. And I probably could
>> get it to be mentioned in Python docs as a best practice (along with
>> Cython/cffi).
> In that case I wouldn't be a fan of it. I hate six, it makes python programming
> more complex, while the whole point of python3 was to clean up some things
> about the language.

 From the code complexity point of view, I don't see the difference 
between the ifdefs being in upstream Python, or in an external library.

>>>>>    - risks restricting developers to the cumbersome common subset - yet another language, essentially
>>>>
>>>> That mostly applies to Python source. In the C extensions, alternatives to
>>>> the removed things have usually been ported to Python 2.6, so not even an
>>>> #ifdef is necesary. The main problem is deciding between bytes and unicode.
>>>
>>> There are already quite a number of ifdefs in the current set of patches, and
>>> this is a fairly simple module..
>>
>> The ifdefs are all* in one header file, which works for tdb & ldb as well.
>> The header would be much smaller if it was just for pytalloc.
> Same difference - it uses magic defines that are ifdeffed in a single place.

Is it the magic that is bothersome?

Most of these they alias one function to another or define a constant.
The others:
- MODULE_INIT_FUNC has one specific use, so I believe it's safe, but I 
can replace it by 4 lines of boilerplate (up from python2's two).
- PY_RICHCMP is relatively magical, yes. Unfortunately this is the way 
Python devs decided comparison is done (it cleans stuff up at the Python 
level, at the expense of C extensions). Also it's not in an ifdef; 
richcmp works the same on both versions.
- Py_RETURN_NOTIMPLEMENTED mirrors Py_RETURN_{NONE,TRUE,FALSE}.
- PyModule_Create could be a function, at the cost of a link-time 
dependency; I don't think it's worth it.

I believe the resulting code is quite readable, do you disagree here?

-- 
Petr Viktorin