Tumbleweed aarch64 build failure #2543

phillxnet · 2023-04-29T11:04:58Z

On x86_64 we have a complete Poetry managed build, but on aarch64 we are failing via:

sh ./build.sh
Retrieving Poetry metadata

The latest version (1.1.15) is already installed.

we are failing to fully install/build our dependencies:

less poetry-install.txt

The currently activated Python version 3.10.10 is not supported by the project (~2.7).
Trying to find and use a compatible version. 
Using python2 (2.7.18)
Creating virtualenv rockstor in /opt/rockstor/.venv
Installing dependencies from lock file
...
Building wheels for collected packages: greenlet
    Building wheel for greenlet (setup.py): started
    Building wheel for greenlet (setup.py): finished with status 'error'
    ERROR: Command errored out with exit status 1:
     command: /opt/rockstor/.venv/bin/python -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-req-build-xOiETe/setup.py'"'"'; __file__='"'"'/tmp/pip-req-build-xOiETe/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-BfPEKh
         cwd: /tmp/pip-req-build-xOiETe/
    Complete output (157 lines):
    running bdist_wheel
    running build
    running build_py
...
copying src/greenlet/tests/_test_extension_cpp.cpp -> build/lib.linux-aarch64-2.7/greenlet/tests
    running build_ext
    building 'greenlet._greenlet' extension
    creating build/temp.linux-aarch64-2.7
    creating build/temp.linux-aarch64-2.7/src
    creating build/temp.linux-aarch64-2.7/src/greenlet
    gcc -fno-strict-aliasing -mbranch-protection=standard -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -DNDEBUG -mbranch-protection=standard -O2 -Wall -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=3 -fstack-protector-strong -funwind-tables -fasynchronous-unwind-tables -fstack-clash-protection -Werror=return-type -g -DOPENSSL_LOAD_CONF -fwrapv -fPIC -I/opt/rockstor/.venv/include/python2.7 -c src/greenlet/greenlet.cpp -o build/temp.linux-aarch64-2.7/src/greenlet/greenlet.o
...
Some compiler warnings, but the same comiler (latest in tumbleweed) works for x86_64
...
   In file included from /usr/include/c++/13/ext/alloc_traits.h:34,
                     from /usr/include/c++/13/bits/basic_string.h:39,
                     from /usr/include/c++/13/string:54,
                     from src/greenlet/greenlet.cpp:10:
    /usr/include/c++/13/bits/alloc_traits.h: In instantiation of ‘struct std::__allocator_traits_base::__rebind<greenlet::PythonAllocator<_greenlet*>, _greenlet*, void>’:
    /usr/include/c++/13/bits/alloc_traits.h:94:11:   required by substitution of ‘template<class _Alloc, class _Up> using std::__alloc_rebind = typename std::__allocator_traits_base::__rebind<_Alloc, _Up>::type [with _Alloc = greenlet::PythonAllocator<_greenlet*>; _Up = _greenlet*]’
    /usr/include/c++/13/bits/alloc_traits.h:228:8:   required by substitution of ‘template<class _Alloc> template<class _Tp> using std::allocator_traits< <template-parameter-1-1> >::rebind_alloc = std::__alloc_rebind<_Alloc, _Tp> [with _Tp = _greenlet*; _Alloc = greenlet::PythonAllocator<_greenlet*>]’
    /usr/include/c++/13/ext/alloc_traits.h:126:65:   required from ‘struct __gnu_cxx::__alloc_traits<greenlet::PythonAllocator<_greenlet*>, _greenlet*>::rebind<_greenlet*>’
    /usr/include/c++/13/bits/stl_vector.h:88:21:   required from ‘struct std::_Vector_base<_greenlet*, greenlet::PythonAllocator<_greenlet*> >’
    /usr/include/c++/13/bits/stl_vector.h:423:11:   required from ‘class std::vector<_greenlet*, greenlet::PythonAllocator<_greenlet*> >’
    src/greenlet/greenlet_thread_state.hpp:115:16:   required from here
    /usr/include/c++/13/bits/alloc_traits.h:70:31: error: static assertion failed: allocator_traits<A>::rebind_alloc<A::value_type> must be A
       70 |                         _Tp>::value,
          |                               ^~~~~
    /usr/include/c++/13/bits/alloc_traits.h:70:31: note: ‘std::integral_constant<bool, false>::value’ evaluates to false
    error: command 'gcc' failed with exit status 1
    ----------------------------------------
    ERROR: Failed building wheel for greenlet
    Running setup.py clean for greenlet
  Failed to build greenlet

The text was updated successfully, but these errors were encountered:

phillxnet · 2023-04-29T11:32:33Z

This looks to be compiler related after-all. It may be that there is no pre-build aarch64 wheel on https://pypi.org/ for greenlet 2.0.1: https://pypi.org/project/greenlet/2.0.1/ (Released Nov7, 2022)
So on our aarch64 build we hit the new Tumbleweed gcc-c++ 13 extra fussiness:

nlohmann/json#3927

N.B. there is now:
2.0.2 available: Released: Jan 28, 2023: https://pypi.org/project/greenlet/

https://github.com/python-greenlet/greenlet

https://github.com/python-greenlet/greenlet/blob/master/CHANGES.rst#202-2023-01-28

Various compilation and standards conformance fixes ...

Specifically the nearest match could be: python-greenlet/greenlet#335

phillxnet · 2023-04-29T12:02:49Z

From our existing poetry.lock file we have our specified (in pyproject.toml) gevent version depending upon - greenlet = ">=0.4.9"

[[package]]
name = "gevent"
version = "1.1.2"

...

[package.dependencies]
greenlet = ">=0.4.9"

phillxnet · 2023-04-29T12:06:18Z

So we likely could just update our poety.lock file to catch the more recently available 2.0.2 greenlet with the suspected compiler fixes.

@FroggyFlox does that sound like the way to go to you? We are way late in testing channel for major changes but this is a secondary dependency - but still.

FroggyFlox · 2023-04-29T12:15:39Z

@phillxnet, if that works then yes! It'll only be a matter of time before this is needed throughout the project anyway.
I can see that our Tumbleweed flavor is doing its job at keeping us up-to-date and ahead of changes ;-).

phillxnet · 2023-04-29T13:39:19Z

Key upstream doc entries for poetry.lock purpose and maintenance:

phillxnet · 2023-04-29T14:22:25Z

Updating the poetry lock file:

rtumbleweed-aarch64:/opt/rockstor # poetry update
Updating dependencies
Resolving dependencies... (131.0s)

Writing lock file

Package operations: 23 installs, 0 updates, 0 removals

  • Installing greenlet (2.0.2)
  • Installing django (1.11.29)
  • Installing gevent (1.1.2)
  • Installing oauthlib (3.1.0)
  • Installing python-engineio (2.3.2)
  • Installing requests (2.27.1)
  • Installing dbus-python (1.2.18)
  • Installing distro (1.6.0)
  • Installing django-braces (1.13.0)
  • Installing django-oauth-toolkit (1.1.2)
  • Installing django-pipeline (1.6.9)
  • Installing djangorestframework (3.9.3)
  • Installing gevent-websocket (0.9.5)
  • Installing gunicorn (19.10.0)
  • Installing huey (2.3.0)
  • Installing mock (1.0.1)
  • Installing psutil (5.9.4)
  • Installing psycogreen (1.0)
  • Installing psycopg2 (2.8.6)
  • Installing python-socketio (1.6.0)
  • Installing pyzmq (19.0.2)
  • Installing supervisor (4.2.4)
  • Installing urlobject (2.1.1)
rtumbleweed-aarch64:/opt/rockstor #

So we have our updated & successfully retrieved (or compiled) greenlet: Installing greenlet (2.0.2)
Note that we have fewer overall installs:
From a similarly updated Tumbleweed but x86_64 we have, from a poetry install resulting from an rpm install.
But using our prior poetry.lock file:

The currently activated Python version 3.10.10 is not supported by the project (~2.7).
Trying to find and use a compatible version. 
Using python2 (2.7.18)
Creating virtualenv rockstor in /opt/rockstor/.venv
Installing dependencies from lock file

Package operations: 29 installs, 0 updates, 0 removals

  • Installing certifi (2021.10.8)
  • Installing chardet (4.0.0)
  • Installing greenlet (2.0.1)
  • Installing idna (2.10)
  • Installing pytz (2022.6)
  • Installing six (1.16.0)
  • Installing urllib3 (1.26.12)
  • Installing django (1.11.29)
  • Installing gevent (1.1.2)
  • Installing oauthlib (3.1.0)
  • Installing python-engineio (2.3.2)
  • Installing requests (2.27.1)
  • Installing dbus-python (1.2.18)
  • Installing distro (1.6.0)
  • Installing django-braces (1.13.0)
  • Installing django-oauth-toolkit (1.1.2)
  • Installing django-pipeline (1.6.9)
  • Installing djangorestframework (3.9.3)
  • Installing gevent-websocket (0.9.5)
  • Installing gunicorn (19.10.0)
  • Installing huey (2.3.0)
  • Installing mock (1.0.1)
  • Installing psutil (5.9.4)
  • Installing psycogreen (1.0)
  • Installing psycopg2 (2.8.6)
  • Installing python-socketio (1.6.0)
  • Installing pyzmq (19.0.2)
  • Installing supervisor (4.2.4)
  • Installing urlobject (2.1.1)

Installing the current project: rockstor (4.5.9)

phillxnet · 2023-04-29T15:09:30Z

Fresh build against the updated poetry.lock file and we have:

cd /opt/rockstor
rm -rf .venv/
sh ./build.sh

with the resulting contents of our poetry-install.txt file (created by build.sh):

rockstor-core/build.sh

Line 23 in cde01f2

    
           /root/.local/bin/poetry install --no-interaction --no-ansi > poetry-install.txt 2>&1

as:

rtumbleweed-aarch64:/opt/rockstor # cat poetry-install.txt
The currently activated Python version 3.10.10 is not supported by the project (~2.7).
Trying to find and use a compatible version. 
Using python2 (2.7.18)
Creating virtualenv rockstor in /opt/rockstor/.venv
Installing dependencies from lock file

Package operations: 29 installs, 0 updates, 0 removals

  • Installing certifi (2021.10.8)
  • Installing chardet (4.0.0)
  • Installing greenlet (2.0.2)
  • Installing idna (2.10)
  • Installing pytz (2022.6)
  • Installing six (1.16.0)
  • Installing urllib3 (1.26.12)
  • Installing django (1.11.29)
  • Installing gevent (1.1.2)
  • Installing oauthlib (3.1.0)
  • Installing python-engineio (2.3.2)
  • Installing requests (2.27.1)
  • Installing dbus-python (1.2.18)
  • Installing distro (1.6.0)
  • Installing django-braces (1.13.0)
  • Installing django-oauth-toolkit (1.1.2)
  • Installing djangorestframework (3.9.3)
  • Installing django-pipeline (1.6.9)
  • Installing gevent-websocket (0.9.5)
  • Installing gunicorn (19.10.0)
  • Installing huey (2.3.0)
  • Installing mock (1.0.1)
  • Installing psutil (5.9.4)
  • Installing psycogreen (1.0)
  • Installing psycopg2 (2.8.6)
  • Installing python-socketio (1.6.0)
  • Installing pyzmq (19.0.2)
  • Installing supervisor (4.2.4)
  • Installing urlobject (2.1.1)

Installing the current project: rockstor (4.5.9)
rtumbleweed-aarch64:/opt/rockstor #

Our prior poetry.lock file had greenlet version 2.0.1, as a dependency of gevent. This version fails to compile under more recent/stricter versions of gcc-13. "poetry.lock" was refresh for its floating package dependencies via: "poetry update".

…_failure Update indirect greenlet dependency to latest version #2543

phillxnet · 2023-04-29T16:08:14Z

Closing as:
Fixed by #2544

This was referenced Apr 29, 2023

Update indirect greenlet dependency to latest version #2543 #2544

Merged

Update changelog on TW aarch64 build fix rockstor/rockstor-rpmbuild#11

Closed

phillxnet added a commit that referenced this issue Apr 29, 2023

Merge pull request #2544 from phillxnet/2543_Tumbleweed_aarch64_build…

ac68e65

…_failure Update indirect greenlet dependency to latest version #2543

phillxnet closed this as completed Apr 29, 2023

phillxnet added this to the First Stable Poetry build milestone Apr 29, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tumbleweed aarch64 build failure #2543

Tumbleweed aarch64 build failure #2543

phillxnet commented Apr 29, 2023

phillxnet commented Apr 29, 2023 •

edited

Loading

phillxnet commented Apr 29, 2023

phillxnet commented Apr 29, 2023 •

edited

Loading

FroggyFlox commented Apr 29, 2023

phillxnet commented Apr 29, 2023

phillxnet commented Apr 29, 2023

phillxnet commented Apr 29, 2023

phillxnet commented Apr 29, 2023

Tumbleweed aarch64 build failure #2543

Tumbleweed aarch64 build failure #2543

Comments

phillxnet commented Apr 29, 2023

phillxnet commented Apr 29, 2023 • edited Loading

phillxnet commented Apr 29, 2023

phillxnet commented Apr 29, 2023 • edited Loading

FroggyFlox commented Apr 29, 2023

phillxnet commented Apr 29, 2023

phillxnet commented Apr 29, 2023

phillxnet commented Apr 29, 2023

phillxnet commented Apr 29, 2023

phillxnet commented Apr 29, 2023 •

edited

Loading

phillxnet commented Apr 29, 2023 •

edited

Loading