[PATCHES] Generate shorter name for extra python files

Tue Sep 19 12:30:57 UTC 2017

On 09/13/2017 04:06 PM, Lukas Slebodnik wrote:
> On (29/08/17 10:35), Petr Viktorin via samba-technical wrote:
>> On 08/27/2017 09:02 AM, Andrew Bartlett wrote:
>>> On Tue, 2017-08-22 at 11:25 +0200, Petr Viktorin via samba-technical
>>> wrote:
>>>>
>>>> I'm sorry for missing this thread; my vacation-mode filtering is too eager.
>>>>
>>>> As for the original design, "extra python" allows building two versions
>>>> of Python by a single `make` invocation. Building for both 2 and 3 at
>>>> once was a requirement to get the Python3 patches in.
>>>> To make this work, I've reused the mechanism Python uses to allow binary
>>>> files for multiple versions to co-exist. The "ABI tag" contains just the
>>>> information needed to distinguish between ABI-incompatible versions.
>>>>
>>>> Note that "/usr/lib64/pkgconfig/pyldb-util36.pc" does not include all
>>>> relevant information: the "m" in "cpython-36m-x86_64-linux-gnu" encodes
>>>> build-time configuration; "dm" would mean a debug build with
>>>> incompatible ABI.
>>>> See [PEP 3149] if you want details.
>>>>
>>>> Using the ABI tag as-is essentially puts the burden of distinguishing
>>>> ABI incompatibilities on Python developers/distributors. It's also quite
>>>> easy to ask Python for it [0], so, hopefully, buildsystems of packages
>>>> that depend on the utils can be made to support this.
>>>>
>>>> That said, I'm not familiar with pkg-config or multilib packaging
>>>> outside of Python, or across distros. This is the Python way to do it;
>>>> please adjust if it makes sense.
>>>>
>>>>
>>>> Another note on the original design intent: generally, the "extra
>>>> python" needs to be version 3, and the "normal" Python could be either 2
>>>> *or* a different version of 3. It's never been tested with py3 as the
>>>> non-extra Python, and I'm sure it doesn't actually work now, but at some
>>>> point it'll be necessary to make the "non-extra" Python be Python 3,
>>>> preferably with the relevant name mangling. Please don't assume that
>>>> "not extra-python" implies Python 2.
>>>>
>>>>
>>>> [PEP 3149]: https://www.python.org/dev/peps/pep-3149/
>>>> [0] import sysconfig; sysconfig.get_config_var('SOABI')
>>>
>>> Thanks for the details.  If you could work with Lumir and Andreas to
>>> propose something that follows the above and meets Lumir's requirements
>>> that would be great.  We can't be the first package to have produced a
>>> python C binding, surely there is some guidance already?
>>
>> Samba is not the first package with a Python binding, but those "_util"
>> libraries are quite unique.
>>
>> General guidance for providing a C API for Python extension modules is
>> provided in Python documentation [0]: roughly, put the function pointers (and
>> a version) in a struct, wrap it in a Capsule object, and provide that as a
>> Python-level object.
>> To use this, there's a PyCapsule_Import helper that imports a module, fetches
>> the attribute, and unwraps it to get the struct.
>>
>> This solves problems with naming/discovery/linking of C-level utilities:
>> everything is found via the extension module.
>> But in Samba's case, it would require changes to all code that currently uses
>> the _util libraries.
>>
>>
>> [0] https://docs.python.org/3/extending/extending.html#providing-a-c-api-for-an-extension-module
>>
> 
> I think LDVERSION (+ fallback to VERSION) should work well in this case
> and does not contain architecture or platform in the string,
>   which is fine for dynamic library.
> 
>    sh$ python3
>    Python 3.6.2 (default, Sep  1 2017, 12:03:48)
>    [GCC 7.1.1 20170802 (Red Hat 7.1.1-7)] on linux
>    Type "help", "copyright", "credits" or "license" for more information.
>    >>> import sysconfig; sysconfig.get_config_var('SOABI')
>    'cpython-36m-x86_64-linux-gnu'
>    >>> from distutils.sysconfig import get_config_var
>    >>> get_config_var('LDVERSION') or get_config_var('VERSION')
>    '3.6m'
>   
>    sh$ python2
>    Python 2.7.13 (default, Aug 16 2017, 12:56:26)
>    [GCC 7.1.1 20170802 (Red Hat 7.1.1-7)] on linux2
>    Type "help", "copyright", "credits" or "license" for more information.
>    >>> import sysconfig; sysconfig.get_config_var('SOABI')
>    >>> from distutils.sysconfig import get_config_var
>    >>> get_config_var('LDVERSION') or get_config_var('VERSION')
>    '2.7'
> 
> 
> Sorry that it took me long time to update patches.

This looks reasonable to me, two nitpicks:

The resulting filename is "libpyldb-util36m.so.1", with no separator 
before the tag. Shouldn't there be one?
If just "-" is used as a separator, the tag can't be a number otherwise 
the buildsystem apparently considers it a version. I suggest adding 
either "-py" in front, or using "-cp" (short for "CPython") to match 
[PEP 425].

And, env['PYTHON_SO_ABI_FLAG'] is no longer necessary.

[PEP 425]: https://www.python.org/dev/peps/pep-0425/#abi-tag

-- 
Petr Viktorin
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0001-Separate-the-Python-ABI-flag-from-the-rest-of-the-so.patch
Type: text/x-patch
Size: 1121 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20170919/94c0fade/0001-Separate-the-Python-ABI-flag-from-the-rest-of-the-so.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 0002-Remove-PYTHON_SO_ABI_FLAG.patch
Type: text/x-patch
Size: 1780 bytes
Desc: not available
URL: <http://lists.samba.org/pipermail/samba-technical/attachments/20170919/94c0fade/0002-Remove-PYTHON_SO_ABI_FLAG.bin>