[PATCHES] Build pytalloc for two Python versions at once, port to py3

[Jelmer Vernooij]

On Thu, Mar 12, 2015 at 06:53:56PM +0100, Petr Viktorin wrote:
> On 03/12/2015 03:41 PM, Jelmer Vernooij wrote:
> >On Thu, Mar 12, 2015 at 03:13:26PM +0100, Petr Viktorin wrote:
> >>On 03/12/2015 02:09 PM, Jelmer Vernooij wrote:
> >>>On Thu, Mar 12, 2015 at 12:08:48PM +0100, Petr Viktorin wrote:
> >>>>On 03/05/2015 10:08 PM, Andrew Bartlett wrote:
> >>>>>On Wed, 2015-03-04 at 15:53 +0100, Jelmer Vernooij wrote:
> >>>>>>On Wed, Mar 04, 2015 at 11:18:08AM +1300, Andrew Bartlett wrote:
> >>>>>>>On Tue, 2015-03-03 at 22:09 +0100, Jelmer Vernooij wrote:
> >>>>>>>>On Mon, Mar 02, 2015 at 12:09:25PM +0100, Petr Viktorin wrote:
> >The onus for adding support of Python3 is on those who care about it, and should
> >not inconvenience the rest of us.
> 
> Alright, then we agree about this bit at least.

> What in the patches proposed here would inconvenience you? Note that with
> these patches I *am* looking specifically at the stand-alone libraries.

I would prefer inlining the ifdefs in the standalone libraries, since
that would be more readable IMHO (not having to switch back and forth
to the wrapper header to see what each define does). Andrew and I seem
to disagree on that though.

> >>>>>  - waiting for things to improve:
> >>>>>   - you suggest is foolish, there is no indication that the python community will
> >>>>>   - and we don't know that someone else will be keen to do the work in the future
> >>>>
> >>>>Also, there's a non-trivial chance that my work on Samba could drive these
> >>>>improvements.
> >>>>It's been suggested a CPython list that I write a porting guide for C code.
> >>>>I'm quite tempted to share what I write for Samba.
> >>>
> >>>Thanks! Upstream improvements to make it easier to support both versions
> >>>would be great.
> >>Just note I wrote *porting guide*, not upstream code.
> >Ah :-(
> >
> >>>I think defines to make it easier to compile 2.6 C code with Python3 belong
> >>>in an upstream header (perhaps hidden behind and #ifdef _COMPAT_PYTHON26 or
> >>>with a deprecation warning), rather than in every Python projects' source
> >>>tree.
> >>I don't think that is going to happen in Python itself. I can however make a
> >>library in the spirit of six/future for C extensions. And I probably could
> >>get it to be mentioned in Python docs as a best practice (along with
> >>Cython/cffi).
> >In that case I wouldn't be a fan of it. I hate six, it makes python programming
> >more complex, while the whole point of python3 was to clean up some things
> >about the language.
> 
> From the code complexity point of view, I don't see the difference between
> the ifdefs being in upstream Python, or in an external library.

That way everybody gets to use the same ifdefs - with documentation,
wider testing - and projects don't have to come up with their own.

> 
> >>>>>   - risks restricting developers to the cumbersome common subset - yet another language, essentially
> >>>>
> >>>>That mostly applies to Python source. In the C extensions, alternatives to
> >>>>the removed things have usually been ported to Python 2.6, so not even an
> >>>>#ifdef is necesary. The main problem is deciding between bytes and unicode.
> >>>
> >>>There are already quite a number of ifdefs in the current set of patches, and
> >>>this is a fairly simple module..
> >>
> >>The ifdefs are all* in one header file, which works for tdb & ldb as well.
> >>The header would be much smaller if it was just for pytalloc.
> >Same difference - it uses magic defines that are ifdeffed in a single place.
> 
> Is it the magic that is bothersome?
> 
> Most of these they alias one function to another or define a constant.
> The others:
> - MODULE_INIT_FUNC has one specific use, so I believe it's safe, but I can
> replace it by 4 lines of boilerplate (up from python2's two).
> - PY_RICHCMP is relatively magical, yes. Unfortunately this is the way
> Python devs decided comparison is done (it cleans stuff up at the Python
> level, at the expense of C extensions). Also it's not in an ifdef; richcmp
> works the same on both versions.
> - Py_RETURN_NOTIMPLEMENTED mirrors Py_RETURN_{NONE,TRUE,FALSE}.
> - PyModule_Create could be a function, at the cost of a link-time
> dependency; I don't think it's worth it.
> 
> I believe the resulting code is quite readable, do you disagree here?

These patches create yet another Python C API for us to use in Samba,
one which is not used by anybody else. This makes it harder for new
and existing contributors to come in and see what is going on,
contribute code or find documentation. There are analoguous problems
for the pure-Python side.

These sorts of compatibility wrappers really belong in upstream Python,
not in every single individual project. Barring that, I guess we'll
have to make due with what we do have...

An nice alternative would be, as Andrew suggested, if we could just include
an extra header that reimplemented the standard Python2 API and called
out to the matching Python3 functions as necessary, or vice versa.
(Similar to what we do in Samba for libreplace). I think this is
pretty hard though.

Cheers,

Jelmer