Increase default python version to 3.8 or higher

[PrimozGodec]

Details

It is not an issue with a specific project but a change request.

When the project does include the.readthedocs.yaml file, documentation is built in Python 3.7 by default. Since many projects already dropped support for Python 3.7 (NumPy, pandas, ...), I think it is time to set the default to 3.8 or higher. Recently I tried to limit the panda's version to >=1.4, and thus documentation build failed since it was built with python 3.7.

You would ask why I do not specify the Python version in .readthecos.ymal. Since my documentation build has multiple subprojects and .readthedocs.yaml does not support that, I cannot.

Expected Result

I wish that readthecos would build documentation on Python 3.8 or 3.9 by default.

[humitos]

Thanks for opening this issue.

By default, we are using build.image: latest which defaults to Python 3.7. I'm not sure we will be able to change this default because it will affect a thousand of projects and I think they will start failing. However, we are discussing about the deprecating these old images and helping old projects with the migration.

I strongly recommend you to use build.os and build.tools.python to specify the versions that suit better for you.

@PrimozGodec

You would ask why I do not specify the Python version in .readthecos.ymal. Since my documentation build has multiple subprojects and .readthedocs.yaml does not support that, I cannot.

Can you expand more on this? I'm not sure to understand why you cannot specify this in the config file.

[PrimozGodec]

Thank you for your detailed answer.

I strongly recommend you to use build.os and build.tools.python to specify the versions that suit better for you.

Where can it be set? You probably mean in .readthedocs.yaml

Can you expand more on this? I'm not sure to understand why you cannot specify this in the config file.

Our project https://readthedocs.org/projects/orange3/ has three subprojects, each of which has its config file. For reference, you can see all three docs here https://github.com/biolab/orange3/tree/master/doc. Each folder is one subproject with its config file.

As far as I understand, if I want to introduce the .readthedocs.yaml file in the repository, it can only point to one config file. By introducing it, I break subproject settings (probably all subprojects would render the same documentation). Correct me if I am wrong, but the only resource on this topic I found is this issue which is not implemented yet: #8811

I would be very happy to have suggestions on moving the build to python 3.8, 3.9, or 3.10.

[humitos]

As far as I understand, if I want to introduce the .readthedocs.yaml file in the repository, it can only point to one config file. By introducing it, I break subproject settings (probably all subprojects would render the same documentation). Correct me if I am wrong, but the only resource on this topic I found is this issue which is not implemented yet: #8811

If I'm understanding correctly, you could use https://github.com/readthedocs/sphinx-multiproject for your use case. I'm not 100% sure it fits, tho. We use it for our own repository and it has been working fine.

[humitos]

Related to this topic, as Sphinx is deprecating Python 3.7 in version 6, and we are installing "latest Sphinx by default" we should do something about it before start breaking the builds. Maybe this is a good moment to start talking about deprecating latest image as well which defaults to Python 3.7.

I'm trying to think about making the less breaking change possibles instead of multiple breaking iterations. I think it would be good to talk about this and make a plan. cc @ericholscher

[stsewd]

Related to this topic, as Sphinx is deprecating Python 3.7 in version 6, and we are installing "latest Sphinx by default" we should do something about it before start breaking the builds.

I think pip should be able to select the latest compatible version (as long as we don't explicitly request the incompatible version)

[PrimozGodec]

If I'm understanding correctly, you could use https://github.com/readthedocs/sphinx-multiproject for your use case. I'm not 100% sure it fits, tho. We use it for our own repository and it has been working fine.

Thank you @humitos for pointing out to sphinx-multiproject. I will try to use it in our project. So if I understand correctly each project would appear as a subproject at readthedocs.org?

[humitos]

Thank you @humitos for pointing out to sphinx-multiproject. I will try to use it in our project. So if I understand correctly each project would appear as a subproject at readthedocs.org?

That will depend on how you set up them. You have to import the same repository multiple times; one per documentation project. Then you can configure them as subprojects on Read the Docs or not. That's up to you.

[humitos]

@stsewd

I think pip should be able to select the latest compatible version (as long as we don't explicitly request the incompatible version)

This is true. However, by keeping latest as default image with Python 3.7 --we will keep lot of users on an old Sphinx version. This will definitely create other issues and multiple support requests.

[jaraco]

I also have this issue. I have several projects that build on Python 3.9+ only, although most still support Python 3.7+. I'd like to avoid the churn across each project of (a) pinning to a later version of Python now and (b) at some point in the future (probably when things start breaking again) pin to an even later version of Python.

The current status quo is for readthedocs to have an arbitrary version of Python with questionable stability. That is, Python 3.7 was a recent release at one point but is a stale version today.

Better would be for RTD to reset the expectation. Instead of letting the "current" version of Python become stale until it's unusable and then bump it several versions, regularly and predictably update the Python version. I'd recommend to update to the latest stable Python at 3-6 months after its release. With this expectation in place, projects could still pin to a specific, older version of Python if their project requires, but the default behavior of RTD would be to support stable, recent versions of Python.

This design would allow me (and I'm guessing hundreds of other users) to rely on the default Python version in RTD and avoid having to declare a specific version at all. RTD should aim for the default settings to work best for new projects created today, no to provide support first for old projects.

[benjaoming]

A new project with build.image: latest will get Python 3.7. From a user's perspective, I would say that this is very unexpected to the point that it's misleading since 3.7 isn't the latest Python version.

[humitos]

@jaraco

I also have this issue. I have several projects that build on Python 3.9+ only, although most still support Python 3.7+. I'd like to avoid the churn across each project of (a) pinning to a later version of Python now and (b) at some point in the future (probably when things start breaking again) pin to an even later version of Python.

You can use:

build:
  tools:
    python: "3"

which will point to the latest stable release:

readthedocs.org/readthedocs/settings/base.py

Lines 609 to 610 in 2ba8fb2

	# Always point to the latest stable release.
	RTD_DOCKER_BUILD_SETTINGS['tools']['python']['3'] = RTD_DOCKER_BUILD_SETTINGS['tools']['python']['3.10']

. Currently, it's 3.10

[humitos]

@benjaoming

A new project with build.image: latest will get Python 3.7. From a user's perspective, I would say that this is very unexpected to the point that it's misleading since 3.7 isn't the latest Python version.

Yeah, build.image refers to the Docker image, not to the Python version. However, we want to remove the stable and latest naming because it leads to confusions. We are in the process of deprecating them and hopefully they won't be available anymore sometime soon. We encourage users to upgrade their projects to use build.os: ubuntu-22.04 instead.

[jaraco]

We encourage users to upgrade their projects to use build.os: ubuntu-22.04 instead.

I'd like to point out that while this recommendation is currently the best recommendation, it will eventually become stale too.

It would be nice if:

• rtd would support build.os: latest
• rtd would support build.tools.python: latest
• make latest the default if nothing is specified
• optionally, require a configuration setting to opt into this behavior (build.prefer-latest=True).

That way, users that specify nothing will be at risk of things changing, but they need not repeatedly monitor/chase the latest config to stay up to date, and they wouldn't need to include many lines of boilerplate across each project. Projects that want stability or need to retain compatibility can pin to older versions, but the default isn't to pin to some arbitrary revision. This approach would match with the behavior that tools like pip and node and GHA use.

[benjaoming]

Related: #8861

[humitos]

@PrimozGodec

When the project does include the.readthedocs.yaml file, documentation is built in Python 3.7 by default.

This is not going to be supported anymore. Read the Docs build system will start requiring a .readthedocs.yaml file to be able to build your documentation. Read more about this in our blog post at https://blog.readthedocs.com/migrate-configuration-v2/

As far as I understand, if I want to introduce the .readthedocs.yaml file in the repository, it can only point to one config file. By introducing it, I break subproject settings (probably all subprojects would render the same documentation).

Now, we support multiple configuration files on the same repository. Read the documentation for this new feature at https://docs.readthedocs.io/en/stable/guides/setup/monorepo.html

I think this issue can be closed now since there is not going to be "default Python version" anymore. However, feel free to comment and re-open if you consider.

[PrimozGodec]

@humitos, thank you for making multiple configuration files available by default. We currently use sphinx-multiproject, but might switch to default implementation.

[humitos]

@PrimozGodec I would appreciate your feedback on that feature once you try it 🙏🏼