Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

setup.py deprecation warning #454

Open
alsora opened this issue Nov 22, 2021 · 28 comments
Open

setup.py deprecation warning #454

alsora opened this issue Nov 22, 2021 · 28 comments

Comments

@alsora
Copy link

alsora commented Nov 22, 2021

Hi, when building Python packages in ROS 2 Galactic the following warning is produced

--- stderr: examples_rclpy_pointcloud_publisher
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
---
Finished <<< examples_rclpy_pointcloud_publisher [1.28s]

is there a plan to change the installation logic to get rid of it?
I'm not expert at all of python packaging, but would it be sufficient to replace the setup.py calls with python -m build ?

@russkel
Copy link

russkel commented Dec 20, 2021

Looks like the Python build process should be moved over to something that follows PEP 517 (https://www.python.org/dev/peps/pep-0517/)

build is probably a good choice.

@ruffsl
Copy link
Member

ruffsl commented Feb 17, 2022

I'm testing some colcon extensions on Ubuntu Jammy 22.04, but noticed the same spammy deprecation notice:

Starting >>> colcon-ros
--- stderr: colcon-cd
/usr/lib/python3/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
---

Guessing we should stop directly executing setup.py files and be call a dedicated python build backend instead?

https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html

@ksuszka
Copy link

ksuszka commented Apr 26, 2022

At our project we try to follow strategy "no warnings during builds/treat all warnings as errors". Is there a way to change our python packages structure/setup.py content/whatever, to remove those warnings? Do we need to change something in our packages or is this an issue only with colcon and we need to live with those warnings till a fix in colcon is implemented?

@russkel
Copy link

russkel commented Apr 26, 2022

You could try and install the master branch of https://github.com/colcon/colcon-python-setup-py via pip and see if that solves the issue.

@ksuszka
Copy link

ksuszka commented Apr 28, 2022

AFAIK I already have the latest version (0.2.7):

root@af77d7851fa2:/ws# pip3 install colcon-python-setup-py -U
Requirement already up-to-date: colcon-python-setup-py in /usr/lib/python3/dist-packages (0.2.7)
Requirement already satisfied, skipping upgrade: colcon-core>=0.6.1 in /usr/lib/python3/dist-packages (from colcon-python-setup-py) (0.8.2)
root@af77d7851fa2:/ws# colcon build --packages-up-to as_ros_license
Starting >>> as_ros_license
--- stderr: as_ros_license                   
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
---
Finished <<< as_ros_license [2.51s]

Summary: 1 package finished [2.78s]
  1 package had stderr output: as_ros_license
root@af77d7851fa2:/ws# 

This issue doesn't occur for setuptools 45.2.0 (which are rather old but still used in the latest official ros:galactic image), however it occurs for setuptools >= 60.

@russkel
Copy link

russkel commented Apr 28, 2022

What's on pypi is behind the master branch.

@ksuszka
Copy link

ksuszka commented Apr 28, 2022

Ok, so another try:

root@23e85568dac8:/ws# pip3 install -U git+https://github.com/colcon/colcon-python-setup-py.git@master
Collecting git+https://github.com/colcon/colcon-python-setup-py.git@master
  Cloning https://github.com/colcon/colcon-python-setup-py.git (to revision master) to /tmp/pip-req-build-mt1uugi5
  Running command git clone -q https://github.com/colcon/colcon-python-setup-py.git /tmp/pip-req-build-mt1uugi5
Requirement already satisfied, skipping upgrade: colcon-core>=0.6.1 in /usr/lib/python3/dist-packages (from colcon-python-setup-py==0.2.7) (0.8.2)
Requirement already satisfied, skipping upgrade: setuptools in /usr/local/lib/python3.8/dist-packages (from colcon-python-setup-py==0.2.7) (62.1.0)
Building wheels for collected packages: colcon-python-setup-py
  Building wheel for colcon-python-setup-py (setup.py) ... done
  Created wheel for colcon-python-setup-py: filename=colcon_python_setup_py-0.2.7-py3-none-any.whl size=11692 sha256=7757e6012eb1146ff7fc3af8077e94a73aba82f21fcf22854dcf7932e2bfc832
  Stored in directory: /tmp/pip-ephem-wheel-cache-dmm0bbmb/wheels/bc/46/d5/da5ce83806c2ab78193e6ee022996337a912af8447706ea89a
Successfully built colcon-python-setup-py
Installing collected packages: colcon-python-setup-py
  Attempting uninstall: colcon-python-setup-py
    Found existing installation: colcon-python-setup-py 0.2.7
    Uninstalling colcon-python-setup-py-0.2.7:
      Successfully uninstalled colcon-python-setup-py-0.2.7
Successfully installed colcon-python-setup-py-0.2.7
root@23e85568dac8:/ws# colcon build --packages-up-to as_ros_license
Starting >>> as_ros_license
--- stderr: as_ros_license
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
---
Finished <<< as_ros_license [2.70s]

Summary: 1 package finished [3.03s]
  1 package had stderr output: as_ros_license
root@23e85568dac8:/ws#

I'm probably missing something.

@ksuszka
Copy link

ksuszka commented Apr 28, 2022

I've prepared reproducible example using docker:
with Dockerfile:

FROM ros:galactic

RUN apt-get update -y && apt-get install -y python3-pip
RUN pip3 install -U setuptools
RUN pip3 install -U git+https://github.com/colcon/colcon-python-setup-py.git@master
RUN mkdir -p /ws/src
WORKDIR /ws
RUN git clone https://github.com/ros2/examples.git --branch galactic /ws/src/examples
RUN colcon build --packages-select examples_rclpy_minimal_publisher

when I run command

docker build . --progress plain --no-cache

I get the same warning at the end:

#11 [8/8] RUN colcon build --packages-select examples_rclpy_minimal_publisher
#11 sha256:3365c76b7b5b832d03997e1d462f1dd7248f09be99b86c49fc69b54590821762
#11 0.780 Starting >>> examples_rclpy_minimal_publisher
#11 3.421 --- stderr: examples_rclpy_minimal_publisher
#11 3.421 /usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
#11 3.421   warnings.warn(
#11 3.421 ---
#11 3.421 Finished <<< examples_rclpy_minimal_publisher [2.64s]
#11 3.443
#11 3.443 Summary: 1 package finished [2.88s]
#11 3.443   1 package had stderr output: examples_rclpy_minimal_publisher
#11 DONE 3.5s

@russkel
Copy link

russkel commented Apr 29, 2022

I'm probably missing something.

No probably not, it was just a suggestion. Good to know it still isn't fixed.

@jacobperron
Copy link
Contributor

jacobperron commented May 24, 2022

IIUC, we can continue to use setuptools as the build backend though we should stop invoking the setup.py CLI (which is deprecated, hence the warning).

I suppose since setuptools supports PEP 517, we could have colcon make appropriate calls to the setuptools library, acting as a build frontend. I think this would require packages built by colcon to supply most of the project metadata in a setup.cfg, instead of the setup.py (reference).

However, it appears the Python community is moving towards using pyproject.toml files for describing package metadata (PEP 621). Also, I think a pyproject.toml may be necessary to leverage modern build front-ends like build, which may make colcon's job easier (e.g. if we replaced setup.py CLI calls with build's CLI). Switching to pyproject.toml files everywhere would probably cause a lot more churn than some other solution, I don't know. It might be worth noting setuptools supports pyproject.toml (instead of setup.cfg), however it is "experimental" (reference). At the very least, it'd probably be nice to have colcon plugin to support packages using a pyproject.toml. It looks like someone has worked on one to support poetry.

Whatever the ultimate solution is, it looks like it may involve updating downstream Python packages still relying on invoking the setup.py CLI. We can keep colcon's current implementation for building Python packages as fallback behavior if the downstream package is not updated to use a static description (e.g. setup.cfg or pyproject.toml).


In the meantime, users can explicitly quiet the deprecation warning if it bothers them with a filter. For example, by setting the following environment variable:

PYTHONWARNINGS=ignore:::setuptools.command.install

@felixdivo
Copy link

In addition, I get /usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools on ROS Humble @ Ubuntu 22.04. I can't find a place where we directly use easy_install though. It this also a problem with colcon?

@jacobperron
Copy link
Contributor

I've noticed the EasyInstallDeprecationWarning deprecation warning, along with PkgResourcesDeprecationWarning, if you use colcon's --symlink-install option. If you want, you could also quiet those warnings by appending to the warnings filter, e.g.

PYTHONWARNINGS=ignore:::setuptools.command.install,ignore:::setuptools.command.easy_install,ignore:::pkg_resources

however, ultimately I think these warnings should all be resolved when colcon can build packages without a setup.py.

@felixdivo
Copy link

Okay, thanks for the confirmation! We'll silence them an wait for a patched colon. 😃

@bperseghetti
Copy link

Still seems to be an open issue, although the package still works as expected after building:

colcon build --packages-select ros2_package --symlink-install 
Starting >>> ros2_package
--- stderr: ros2_package                  
/home/ben/.local/lib/python3.10/site-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/user/.local/lib/python3.10/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/home/user/.local/lib/python3.10/site-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 0.1.43ubuntu1 is an invalid version and will not be supported in a future release
  warnings.warn(
/home/user/.local/lib/python3.10/site-packages/pkg_resources/__init__.py:123: PkgResourcesDeprecationWarning: 1.1build1 is an invalid version and will not be supported in a future release
  warnings.warn(
---
Finished <<< ros2_package [2.47s]

Summary: 1 package finished [2.75s]
  1 package had stderr output: ros2_package

@dirk-thomas is anyone currently working on this or has there been any determination on the desired direction to head in resolving it (so we can help contribute to that effort)?

@jacobperron
Copy link
Contributor

I believe @cottsay is working on this issue.

@FranekStark
Copy link

any news on this? :)

@cottsay
Copy link
Member

cottsay commented Sep 29, 2022

This is a complicated change, and I don't have a lot of good news.

The new APIs that the Python folks are pushing us towards will regress the experience here in multiple ways.

  1. The new APIs will force us to create Python wheels and then extract them, meaning that there will be a compression/decompression cycle affecting build performance.
  2. The new APIs don't give us a way to inspect metadata like package name and dependencies, meaning there will be some packages using pyproject.toml that we won't be able to discover or build.
  3. Though there are currently changes proposed to alleviate this, the entire "develop" scenario is currently unsupported meaning that symlink installs will go from "sometimes working" to never working.

I know that folks hate warning messages. I hear you. Implementing this change right now with the tools and APIs available will likely do more harm than good. Please take comfort in the fact that even the new pyproject.toml APIs which use setuptools under the hood are relying on setuptools itself to suppress it's own warning and invoke itself similar to how we invoke it here.

If you simply can't ignore the warning, please familiarize yourself with the PYTHONWARNINGS environment variable, which you can use to specifically suppress certain warnings until we can arrive at a solution that doesn't regress things so badly.

@ralwing
Copy link

ralwing commented Nov 14, 2022

Quite a similar discussion is at ament_cmake repo:
ament/ament_cmake#382

@moyutianque
Copy link

moyutianque commented Nov 17, 2022

I tried to downgrad setuptools to 58.2.0 by
pip3 install setuptools==58.2.0
then colcon build --symlink-install works does not show error but
ros2 test still has the following error

Finished <<< examples_rclpy_pointcloud_publisher [0.96s]
--- stderr: launch_testing_examples                       

=============================== warnings summary ===============================
launch_testing_examples/check_msgs_launch_test.py: 2 warnings
launch_testing_examples/check_multiple_nodes_launch_test.py: 2 warnings
launch_testing_examples/check_node_launch_test.py: 2 warnings
launch_testing_examples/hello_world_launch_test.py: 2 warnings
launch_testing_examples/record_rosbag_launch_test.py: 2 warnings
launch_testing_examples/set_param_launch_test.py: 2 warnings
  Warning: There is no current event loop

test/test_flake8.py::test_flake8
test/test_flake8.py::test_flake8
  Warning: SelectableGroups dict interface is deprecated. Use select.

-- Docs: https://docs.pytest.org/en/stable/warnings.html
---
Finished <<< launch_testing_examples [16.5s]

Summary: 22 packages finished [18.6s]
  10 packages had stderr output: examples_rclpy_executors examples_rclpy_guard_conditions examples_rclpy_minimal_action_client examples_rclpy_minimal_action_server examples_rclpy_minimal_client examples_rclpy_minimal_publisher examples_rclpy_minimal_service examples_rclpy_minimal_subscriber examples_rclpy_pointcloud_publisher launch_testing_examples

@Feedback02
Copy link

I tried to downgrad setuptools to 58.2.0 by pip3 install setuptools==58.2.0 then colcon build --symlink-install works does not show error but ros2 test still has the following error

Finished <<< examples_rclpy_pointcloud_publisher [0.96s]
--- stderr: launch_testing_examples                       

=============================== warnings summary ===============================
launch_testing_examples/check_msgs_launch_test.py: 2 warnings
launch_testing_examples/check_multiple_nodes_launch_test.py: 2 warnings
launch_testing_examples/check_node_launch_test.py: 2 warnings
launch_testing_examples/hello_world_launch_test.py: 2 warnings
launch_testing_examples/record_rosbag_launch_test.py: 2 warnings
launch_testing_examples/set_param_launch_test.py: 2 warnings
  Warning: There is no current event loop

test/test_flake8.py::test_flake8
test/test_flake8.py::test_flake8
  Warning: SelectableGroups dict interface is deprecated. Use select.

-- Docs: https://docs.pytest.org/en/stable/warnings.html
---
Finished <<< launch_testing_examples [16.5s]

Summary: 22 packages finished [18.6s]
  10 packages had stderr output: examples_rclpy_executors examples_rclpy_guard_conditions examples_rclpy_minimal_action_client examples_rclpy_minimal_action_server examples_rclpy_minimal_client examples_rclpy_minimal_publisher examples_rclpy_minimal_service examples_rclpy_minimal_subscriber examples_rclpy_pointcloud_publisher launch_testing_examples

yeah i have the same issue with Humble with ubuntu 22.04.1 even after trying what you did

@ijnek
Copy link
Contributor

ijnek commented Nov 19, 2022

I tried to downgrad setuptools to 58.2.0 by pip3 install setuptools==58.2.0 then colcon build --symlink-install works does not show error but ros2 test still has the following error

Finished <<< examples_rclpy_pointcloud_publisher [0.96s]
--- stderr: launch_testing_examples                       

=============================== warnings summary ===============================
launch_testing_examples/check_msgs_launch_test.py: 2 warnings
launch_testing_examples/check_multiple_nodes_launch_test.py: 2 warnings
launch_testing_examples/check_node_launch_test.py: 2 warnings
launch_testing_examples/hello_world_launch_test.py: 2 warnings
launch_testing_examples/record_rosbag_launch_test.py: 2 warnings
launch_testing_examples/set_param_launch_test.py: 2 warnings
  Warning: There is no current event loop

test/test_flake8.py::test_flake8
test/test_flake8.py::test_flake8
  Warning: SelectableGroups dict interface is deprecated. Use select.

-- Docs: https://docs.pytest.org/en/stable/warnings.html
---
Finished <<< launch_testing_examples [16.5s]

Summary: 22 packages finished [18.6s]
  10 packages had stderr output: examples_rclpy_executors examples_rclpy_guard_conditions examples_rclpy_minimal_action_client examples_rclpy_minimal_action_server examples_rclpy_minimal_client examples_rclpy_minimal_publisher examples_rclpy_minimal_service examples_rclpy_minimal_subscriber examples_rclpy_pointcloud_publisher launch_testing_examples

yeah i have the same issue with Humble with ubuntu 22.04.1 even after trying what you did

This is an unrelated flake8 issue that is tracked in https://bugs.launchpad.net/ubuntu/+source/python-flake8/+bug/1968072, so it has nothing to do with this ticket.

@cottsay
Copy link
Member

cottsay commented May 6, 2023

We'll be discussing this topic at the Infrastructure Community Meeting on May 17: https://discourse.ros.org/t/infrastructure-community-meeting-2023-05-17/31268

@cottsay
Copy link
Member

cottsay commented Jun 16, 2023

I've issued a Call For Testing on a prototype package based on the PEPs discussed here: https://discourse.ros.org/t/call-for-testing-standards-based-python-packaging-with-colcon/32008

If you're interested in seeing these deprecation warnings disappear, please take a moment to provide feedback.

@Flova
Copy link

Flova commented Sep 26, 2023

Whats the current state of this issue?

@cottsay
Copy link
Member

cottsay commented Sep 26, 2023

The prototype is available and working, but there are problems with symlink installs and general performance. PEP 660 with setuptools is currently ignoring data_files, which is the mechanism that ROS packages use to install things like package manifests and launch files. That alone is a blocker.

If you're interested in this landing sooner, please consider trying the prototype package and providing feedback.

@TheMarksniper
Copy link

The prototype is available and working, but there are problems with symlink installs and general performance. PEP 660 with setuptools is currently ignoring data_files, which is the mechanism that ROS packages use to install things like package manifests and launch files. That alone is a blocker.

If you're interested in this landing sooner, please consider trying the prototype package and providing feedback.

Is there any way to download this prototype via pip?

@cottsay
Copy link
Member

cottsay commented Jan 23, 2024

Is there any way to download this prototype via pip?

You can target the git branches directly using pip, but I don't think it's a good idea at all. Pip has a tendency to tangle up your system and it might make it difficult to revert back to the non-prototype packages.

@rafa-martin
Copy link

As in PR #626, it would be beneficial to suppress this error by default as well:

/usr/lib/python3/dist-packages/setuptools/command/easy_install.py:158: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
---

_PYTHON_CMD = [
sys.executable,
'-W',
'ignore:setup.py install is deprecated',
]

The updated implementation could look like this:

_PYTHON_CMD = [
    sys.executable,
    '-W',
    'ignore:setup.py install is deprecated',
    '-W',
    'ignore:easy_install command is deprecated',
]

Would it make sense to suppress this error too?"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests