build packages for multiple python targets #3945

totaam · 2023-07-28T13:02:26Z

Fedora and RHEL can install multiple versions of python3.
Only the primary one has all the packages we usually need in the repositories (gobject, etc), but it should be possible to make builds without those and install the missing dependencies via pip.
This would allow us to continue to support some RHEL 8 / 9 with xpra v6 and later.

The text was updated successfully, but these errors were encountered:

totaam · 2023-07-28T14:35:43Z

Mostly done in the commits above, at least for Fedora.
The repo-build-scripts will need updating to support building multiple sets of packages for each python version.
This increases the number of packages we have to build for non-standard-python builds: pygobject3, pycairo, python3-setuptools.
Some packages don't build yet because they would require more dependencies themselves: pycuda, aioquic.

One remaining issue is that the new package split (#3802) includes configuration files in multiple packages.
This prevents parallel installation.
Fixing this is hard:

some files could be moved to a xpra-filesystem type of package, ie: /etc/xpra, /usr/libexec/ and /usr/share/, tmpfiles.d, sysusers.d, etc
some could be excluded in the non-default-python package? ie: /lib/systemd/ (bearing in mind that some distributions will not have a 'default-python' build at all..)
others are more difficult: /etc/xpra/cuda.conf, etc

let the python3 version check do its thing

also silence warnings when called from rpmspec when the python version needed is not installed yet

* replace '%pycached' with glob on RHEL8 * make all packages python-versioned

totaam · 2023-07-29T19:54:37Z

Now also available for almalinux 8 and 9.
The only remaining issue is the multiple python version installation conflict.

we cannot mix with pure 'python3' packages

totaam · 2023-07-30T12:17:41Z

Now usable and parallel installable thanks to the new common xpra-filesystem package.

ie on Fedora 38, we can install 3 different python builds at the same time (xpra is the default which is python-3.11):

dnf install python3.10-xpra python3.12-xpra xpra

Remaining issues:

the xpraforwarder cups script should be to installed as part of xpra-filesystem but let xpra-server symlink it in the cups driver location. We can't easily refcount it, so removal is more tricky.
run_scaled and xpra_launcher are always installed.. so we should give a more helpful error message if the actual implementation cannot be found. ie: install xpra-client-gtk or use the matching python interpreter
re-launch /usr/bin scripts with a different interpreter? change shebang?
document that these packages are bare-bones and a lot of libraries will be missing - packaged, but not fully supported
test on rocky / alma / oracle

totaam · 2023-08-01T14:45:59Z

Had to remove pygobject-* and python-gobject* due to a conflict reported with the system version.
Perhaps this will require building the exact same version as the system library on all distributions? This would be a major PITA.

so leave the python3 version used for building as shebang

totaam · 2023-08-15T14:47:28Z

This will do.
I've tried hard to make RHEL and clones upgrade from earlier version to the new python3.11-xpra packages, to no avail.
For some strange reason, a dnf update only brings in xpra-filesystem!
So one has to run dnf install python3.11-xpra to get the new version.

totaam · 2023-08-22T13:41:59Z

Everything seems to work OK now. Tested both clean installations and upgrades from 5.0

RHEL 8 and 9 get a package called xpra which is actually a python3.11-xpra under the hood.

totaam · 2023-08-25T13:58:20Z

Upgrades were blocked for RHEL 8 and clones because of a missing python3.11-pycuda.
Fixed in 128aaf5 + 47af866

Also fixed Python 3.12 pycuda builds: 1da51d4

totaam · 2023-09-30T07:53:44Z

There are still some files installed as part of xpra-server and I thought that these would conflict, but somehow rpm is happy to have the same file provided by 3 different packages (same file contents):

$ rpm -qf /usr/libexec/xpra/auth_dialog
python3.12-xpra-server-6.0-10.r34426.fc37.x86_64
xpra-server-6.0-10.r34426.fc37.x86_64
python3.10-xpra-server-6.0-10.r34426.fc37.x86_64

totaam · 2024-01-17T10:36:19Z

Blocker on Almalinux 8.9:

$ sudo dnf update
Last metadata expiration check: 0:14:36 ago on Wed 17 Jan 2024 04:45:02 EST.
Error: 
 Problem: cannot install both python3.11-gobject-3.44.1-2.el8.x86_64 from xpra-beta and python3.11-gobject-3.44.1-1.el8.x86_64 from @System
  - package python3.11-gobject-devel-3.44.1-1.el8.x86_64 from @System requires python3.11-gobject(x86-64) = 3.44.1-1.el8, but none of the providers can be installed
  - cannot install the best update candidate for package python3.11-gobject-3.44.1-1.el8.x86_64
  - problem with installed package python3.11-gobject-devel-3.44.1-1.el8.x86_64
(try to add '--allowerasing' to command line to replace conflicting packages or '--skip-broken' to skip uninstallable packages or '--nobest' to use not only best candidate packages)

Not sure why I managed to update on CentOS stream9 and rocky 9...

totaam · 2024-01-18T03:49:27Z

New problems with the latest aioquic and pylsqpack on RHEL9 (and maybe others):

dnf install python3-aioquic --nobest
(..)
 Problem: cannot install the best candidate for the job
  - nothing provides (python3.9dist(pylsqpack) < 0.4 with python3.9dist(pylsqpack) >= 0.3.3) needed by python3-aioquic-0.9.25-1.el9.x86_64 from xpra-beta
  - nothing provides python3.9dist(pyopenssl) >= 22 needed by python3-aioquic-0.9.25-1.el9.x86_64 from xpra-beta
  - nothing provides python3.9dist(service-identity) >= 23.1 needed by python3-aioquic-0.9.25-1.el9.x86_64 from xpra-beta

The beta repository for RHEL9 is meant for python3.11 prefixed packages, but the updated unprefixed python3 builds should still work.

First odd thing is that these packages it is complaining about are actually available:

dnf install python3-pylsqpack python3-pyOpenSSL
(..)
python3-pyOpenSSL          noarch          21.0.0-1.el9             epel           90 k
python3-pylsqpack          x86_64          0.3.18-1.el9             xpra           83 k

The same package versions do install just fine on Fedora 39..
I should get points for circling back to my own comment on an existing answer: https://superuser.com/a/557112/58851

totaam · 2024-01-18T07:13:29Z

Despite being the same version and using the default python3 interpreter, the packages for almalinux 9 were built from the v5.x branch, whereas the other distros got them from git master! (the dangers of working on multiple branches at the same time)

The v5.x branch needed AutoReq: 0 and its python relatives: 51cfb4e
(don't ask me why the python autoreq gets confused since these rpm specfiles don't use any python3 prefix shenanigans)

see #3945 (comment) for details

totaam · 2024-01-18T08:22:58Z

Going back to #3945 (comment) and pygobject3, this all stems from the quite bizarre decision of shipping a binary python3-gobject package but not the python3-gobject-devel.
Only affects upgrades from a previous v6 beta installation (remove then re-install works), so I'm not going to worry too much about it..

totaam · 2024-01-20T07:30:38Z

We now even have python3.13 builds for Fedora 39:

$ sudo dnf install python3.13-xpra
Xpra Beta 39 - x86_64                                                                                                                                                                                                                     151 kB/s | 366 kB     00:02    
Last metadata expiration check: 0:00:01 ago on Sat 20 Jan 2024 14:29:13 +07.
Dependencies resolved.
==========================================================================================================================================================================================================================================================================
 Package                                                                  Architecture                                        Version                                                           Repository                                                           Size
==========================================================================================================================================================================================================================================================================
Installing:
 python3.13-xpra                                                          x86_64                                              6.0-10.r34972.fc39                                                xpra-beta                                                            14 k
Installing dependencies:
 python3.13-xpra-client                                                   x86_64                                              6.0-10.r34972.fc39                                                xpra-beta                                                           132 k
 python3.13-xpra-client-gtk3                                              x86_64                                              6.0-10.r34972.fc39                                                xpra-beta                                                           616 k
 python3.13-xpra-codecs                                                   x86_64                                              6.0-10.r34972.fc39                                                xpra-beta                                                           586 k
 python3.13-xpra-common                                                   x86_64                                              6.0-10.r34972.fc39                                                xpra-beta                                                           1.7 M
 python3.13-xpra-server                                                   x86_64                                              6.0-10.r34972.fc39                                                xpra-beta                                                           963 k
 xpra-filesystem                                                          noarch                                              6.0-10.r34972.fc39                                                xpra-beta                                                           6.2 M
Installing weak dependencies:
 cups-pdf                                                                 x86_64                                              3.0.1-19.fc39                                                     fedora                                                               42 k
 gnome-icon-theme                                                         noarch                                              3.12.0-20.fc39                                                    fedora                                                              9.9 M
 gstreamer1-plugins-ugly                                                  x86_64                                              1:1.22.8-1.fc39                                                   rpmfusion-free-updates                                               86 k
 gtk3-immodule-xim                                                        x86_64                                              3.24.39-1.fc39                                                    updates                                                              22 k
 mesa-libOSMesa                                                           x86_64                                              23.3.3-1.fc39                                                     updates                                                             3.2 M
 pulseaudio-utils                                                         x86_64                                              16.1-5.fc39                                                       fedora                                                               82 k
 python3.13-pyu2f                                                         x86_64                                              0.1.5-3                                                           xpra-beta                                                            67 k
 python3.13-xpra-audio                                                    x86_64                                              6.0-10.r34972.fc39                                                xpra-beta                                                            97 k
 python3.13-xpra-x11                                                      x86_64                                              6.0-10.r34972.fc39                                                xpra-beta                                                           1.1 M
 qrencode                                                                 x86_64                                              4.1.1-5.fc39                                                      fedora                                                               25 k

Transaction Summary
==========================================================================================================================================================================================================================================================================
Install  17 Packages

totaam added enhancement New feature or request packaging linux labels Jul 28, 2023

totaam added a commit that referenced this issue Jul 28, 2023

#3945 build packages for 'PYTHON3' env var version

05211d2

totaam added a commit that referenced this issue Jul 28, 2023

#3945 support build for multiple PYTHON3 targets

28841aa

totaam mentioned this issue Jul 28, 2023

5.0 packaging updates #3802

Closed

totaam added a commit that referenced this issue Jul 29, 2023

#3945 build python3.10 and python3.12 packages for Fedora

060cd40

totaam added a commit that referenced this issue Jul 29, 2023

#3945 build python3.11 packages for RHEL

285ff6b

totaam added a commit that referenced this issue Jul 29, 2023

#3945 no need to block specific distros

1df3c92

let the python3 version check do its thing

totaam added a commit that referenced this issue Jul 29, 2023

#3945 only override when needed

1573054

also silence warnings when called from rpmspec when the python version needed is not installed yet

totaam added a commit that referenced this issue Jul 29, 2023

#3945 use madvise

d2a65d6

totaam added a commit that referenced this issue Jul 29, 2023

#3945 RHEL fixes

93f01fc

* replace '%pycached' with glob on RHEL8 * make all packages python-versioned

totaam added a commit that referenced this issue Jul 30, 2023

#3945 we always need pillow

df49b59

totaam added a commit that referenced this issue Jul 30, 2023

#3945 always use prefix

9fa61f1

we cannot mix with pure 'python3' packages

totaam added a commit that referenced this issue Jul 30, 2023

#3945 split into new common 'filesystem' package

c8798f6

totaam added a commit that referenced this issue Jul 30, 2023

#3945 use the same spec file for pycairo

480a33c

totaam added a commit that referenced this issue Aug 1, 2023

#3945 try to ensure downgrades from v6 work more smoothly

816ff93

totaam added a commit that referenced this issue Aug 1, 2023

#3945 'filesystem' package conflicts with older 'common' package

0caa26f

totaam added a commit that referenced this issue Aug 3, 2023

#3945 ensure the main scripts are version agnostic

deee8b1

This was referenced Aug 4, 2023

make it easier to disable almost everything #3953

Closed

v5.1 features tracker ticket #3954

Open

totaam added a commit that referenced this issue Aug 8, 2023

#3945 more common scripts moved to 'xpra-filesystem'

4d6f6fe

totaam added a commit that referenced this issue Aug 12, 2023

#3945 RHEL and clones don't have a valid 'python3'

8212d9d

so leave the python3 version used for building as shebang

totaam added a commit that referenced this issue Aug 14, 2023

#3945 enable upgrades to v6 on RHEL and clones

91900b9

totaam added a commit that referenced this issue Aug 14, 2023

#3945 'obsoletes' should trigger removal of older versions

4003edb

totaam added a commit that referenced this issue Aug 14, 2023

#3945 simplify and manage all dependencies / requires ourselves

4c934fd

totaam closed this as completed Aug 15, 2023

totaam added a commit that referenced this issue Aug 15, 2023

#3945 more helpful error message

27ad5ad

totaam added a commit that referenced this issue Aug 15, 2023

#3945 more helpful error message

5986a59

totaam added a commit that referenced this issue Aug 22, 2023

#3945 simplify? only a single package should obsolete another one

a64a180

totaam added a commit that referenced this issue Aug 22, 2023

#3945 make it possible to build multiple python versions in parallel

4af0ed8

totaam added a commit that referenced this issue Oct 2, 2023

#3945 build more packages for non-default python

ae09c09

totaam added a commit that referenced this issue Oct 10, 2023

#3945 try to ensure downgrades from v6 work more smoothly

4c22188

totaam mentioned this issue Oct 15, 2023

RPM packaging of Xpra for Fedora #4035

Closed

5 tasks

totaam added a commit that referenced this issue Nov 17, 2023

#3945 the headers are not tied to the python version

5fb42a9

totaam reopened this Jan 17, 2024

totaam added a commit that referenced this issue Jan 18, 2024

I no longer trust the automagic dependencies

e39bbd5

see #3945 (comment) for details

totaam closed this as completed Jan 18, 2024

totaam mentioned this issue May 3, 2024

respect python PATH #4213

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

build packages for multiple python targets #3945

build packages for multiple python targets #3945

totaam commented Jul 28, 2023

totaam commented Jul 28, 2023 •

edited

Loading

totaam commented Jul 29, 2023

totaam commented Jul 30, 2023

totaam commented Aug 1, 2023 •

edited

Loading

totaam commented Aug 15, 2023

totaam commented Aug 22, 2023

totaam commented Aug 25, 2023 •

edited

Loading

totaam commented Sep 30, 2023

totaam commented Jan 17, 2024

totaam commented Jan 18, 2024 •

edited

Loading

totaam commented Jan 18, 2024 •

edited

Loading

totaam commented Jan 18, 2024

totaam commented Jan 20, 2024

build packages for multiple python targets #3945

build packages for multiple python targets #3945

Comments

totaam commented Jul 28, 2023

totaam commented Jul 28, 2023 • edited Loading

totaam commented Jul 29, 2023

totaam commented Jul 30, 2023

totaam commented Aug 1, 2023 • edited Loading

totaam commented Aug 15, 2023

totaam commented Aug 22, 2023

totaam commented Aug 25, 2023 • edited Loading

totaam commented Sep 30, 2023

totaam commented Jan 17, 2024

totaam commented Jan 18, 2024 • edited Loading

totaam commented Jan 18, 2024 • edited Loading

totaam commented Jan 18, 2024

totaam commented Jan 20, 2024

totaam commented Jul 28, 2023 •

edited

Loading

totaam commented Aug 1, 2023 •

edited

Loading

totaam commented Aug 25, 2023 •

edited

Loading

totaam commented Jan 18, 2024 •

edited

Loading

totaam commented Jan 18, 2024 •

edited

Loading