Can not build python plugin for Debian package

[unxed]

Build script is trying to fetch debugpy over the Internet, but it is not allowed by Debian build process:

WARNING: Retrying (Retry(total=0, connect=None, read=None, redirect=None, status=None)) after connection broken by 'NewConnectionError('<pip._vendor.urllib3.connection.HTTPSConnection object at 0x7fadfbabcb90>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution')': /simple/debugpy/
ERROR: Could not find a version that satisfies the requirement debugpy (from versions: none)
ERROR: No matching distribution found for debugpy
make[4]: *** [python/CMakeFiles/python.dir/build.make:83: python/incpy/far2lcffi.py] Error 1

@m32

[unxed]

The required package is already in Debian, you just need to use the system package if it is available.

[m32]

I don't really see a solution for building the far2l package with a python virtual environment in the form that is done using cmake.

When I created the plugin, I did not want to have one Python for everything, but dedicated Pythons for subsequent tasks, hence the idea with virtualenv. If I am not mistaken, during the installation of the Debian package, you can only say which packages it uses and I think you cannot create a virtual Python environment during the installation of the Debian package.

The only solution I see is to change the python plugin.
It should proxy calls to another file (.so) and will only start if the plugin parameters contain the address of a useful python virtual environment created by the current user with the missing libraries installed (cffi, debugpy, adbutils, ...).
Then the user will decide how to install the missing python packages.

[mitya57]

The Debian build environment will have a working python3 in $PATH, and with that python3 the cffi and debugpy modules will be importable. Also it will have the development headers of that Python version in include path.

The only thing which is not available is adbutils. But is it required or optional?

And when the package is built, it will have runtime dependencies on all modules and on the Python library version which it was built against.

What else is needed that Debian does not provide?

[m32]

only a working cffi is required, rest is or can be made optional

[mitya57]

Okay, so my wishlist (as a Debian developer) is changing the build system to not try to install packages from the internet if they are already installed.

[m32]

Please check it out the fix: #2420

far2l uses python interpreter from system, cffi is used from the system packages, virtualenv is not used, no packages are downloaded from pypi

[m32]

@mitya57 @unxed

as suggested I changed the way the python plugin is built, without downloading anything from pypi or using external tools I have built a debian package and python is in it

[unxed]

@alexmyczko

[alexmyczko]

please merge? please release?

[m32]

merge before release

[m32]

another fix #2435 - python included in packaging build

[shmuz]

@m32
After pulling far2l containing your PR and building far2l and plugins I get the following:

[PYTHON 436]: PythonHolder@203 - pluginpath: /home/shmuel/far2l/_build/install/Plugins/python/plug, python library used:/usr/lib/x86_64-linux-gnu/libpython3.10.so
[PYTHON 440]: ThreadProc@168 - initial sys.path=import sys
sys.path.insert(1, '/home/shmuel/far2l/_build/install/Plugins/python/plug')
Traceback (most recent call last):
  File "/home/shmuel/far2l/_build/install/Plugins/python/plug/far2l/__init__.py", line 1, in <module>
    from .pluginmanager import PluginManager
  File "/home/shmuel/far2l/_build/install/Plugins/python/plug/far2l/pluginmanager.py", line 56, in <module>
    from .far2lcffi import ffi
  File "/home/shmuel/far2l/_build/install/Plugins/python/plug/far2l/far2lcffi.py", line 4, in <module>
    import cffi
ModuleNotFoundError: No module named 'cffi'
[PYTHON 484]: ThreadProc@178 - Failed to load "far2l"

Your advice would be appreciated.

[m32]

python plugin uses only packages from system, just run apt install python3-cffi

[unxed]

Can we consider this problem closed now and ship python inside Debian package?

[alexmyczko]

fine for me however the pkg still is broken in sid and i am waiting for a new release

[unxed]

@m32 I still see changes. Please let us know as soon as it's ready for release.

[m32]

I don't plan to make any more changes.
I generated a .deb package and installed it on the new machine, the plugin starts correctly.

[unxed]

@m32 still a little delay on start if far2l is built with Python plugin. If the plan to include it in Debian, can some kind of lazy loading be implemented?

UPD: according to @shmuz test, start time is:
Without python: 135 ms
With python: 408 ms
Tests on another systems will give different numbers of course.

Idea by @akruphi

[m32]

Unfortunately I can't do anything about it, python has to load in the main thread of the program and the trick of starting it in the background didn't work. Even if it worked, the problems would come out later - I expect that because not every library will be able to work correctly in a thread other than the main message loop (sqlite, gtk, wx?)