Strategy for pinning dependencies

[ericholscher]

Overview

This is a ticket to discuss the best way to pin dependencies for Sphinx. The current status appears to be the latest Sphinx 3.x and 4.x release is pinning docutils<0.17. This will work around the existing changes in docutils until they can be tested (#9088).

This raises the question of other dependencies that might change HTML output and break users experience with Sphinx. I can think of two major categories of these changes:

• Direct dependencies that change HTML output (eg. pygments & docutils)
• Transitive dependencies that have the same outcome (I don't know what these would be, but they likely exist)

Direct Dependencies

In terms of direct dependencies, I think we probably want to pin to the latest minor version (eg. <2.9 for Pygments, which is currently at 2.8.1). This will handle most of the obvious issues.

Transitive Dependencies

The trickier thing is what to do with transitive dependencies. I think there are two options here:

1. We can work with our dependencies to pin their dependencies (and so on down the chain :D)
2. We can research the obvious things that might break (have any broken in the past?) and pin them directly on Sphinx

I think we should opt for #1, but do #2 as a backup if needed.

Next Steps

Since we currently have docutils pinned, we need to see what other Sphinx dependencies need to be pinned, and work to get those first. Anything that we pin, we need to make sure we have a process for testing changes to and upgrading them with releases, so we need to work alongisde #9088 on this process.

[j1elo]

I'd like to bring attention to this, as today out of the blue previously working ReadTheDocs builds started to fail on me for no apparent reason. Turns out sphinxcontrib-applehelp (and other similar dependencies of Sphinx) over time have changed from below our feet and broke previously functioning installs.

My error was one that it seem has bitten more people in the past, so it won't come as a surprise I think:

Running Sphinx v4.5.0

[...]

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/doc-kurento/envs/latest/lib/python3.11/site-packages/sphinx/cmd/build.py", line 272, in build_main
    app = Sphinx(args.sourcedir, args.confdir, args.outputdir,
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/docs/checkouts/readthedocs.org/user_builds/doc-kurento/envs/latest/lib/python3.11/site-packages/sphinx/application.py", line 219, in __init__
    self.setup_extension(extension)
  File "/home/docs/checkouts/readthedocs.org/user_builds/doc-kurento/envs/latest/lib/python3.11/site-packages/sphinx/application.py", line 380, in setup_extension
    self.registry.load_extension(self, extname)
  File "/home/docs/checkouts/readthedocs.org/user_builds/doc-kurento/envs/latest/lib/python3.11/site-packages/sphinx/registry.py", line 441, in load_extension
    raise VersionRequirementError(
sphinx.errors.VersionRequirementError: The sphinxcontrib.applehelp extension used by this project needs at least Sphinx v5.0; it therefore cannot be built with this version.

requirements.txt looked like this:

Sphinx==4.5.0
sphinx-rtd-theme==1.0.0

but, after seeing similar reports such as #11890 (comment) or googleapis/sphinx-docfx-yaml#344, I had to make it look like this instead to pin all sphinxcontrib dependencies:

Sphinx==4.5.0
sphinx-rtd-theme==1.0.0

sphinxcontrib-applehelp<=1.0.4
sphinxcontrib-devhelp<=1.0.2
sphinxcontrib-jsmath<=1.0.1
sphinxcontrib-htmlhelp<=2.0.1
sphinxcontrib-serializinghtml<=1.1.5
sphinxcontrib-qthelp<=1.0.3

Albeit I think that discovering and setting what are the valid ranges for its own dependencies is a job that Sphinx itself should do.