-
-
Notifications
You must be signed in to change notification settings - Fork 2.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Circular dependencies between Sphinx
and sphinxcontrib-*
#11567
Comments
After reading up on the conversation starting here, it seems like the cleanest workaround is the one described here, which would allow one to manually remove the back-dependencies of This should work 100% cleanly for all Bazel users who know that they will only be using I'll leave it up to discussion here whether having the |
Hi Bruno, Thank you for your detailed issue. I have no experience with Bazel or In What would be a Bazel-compatible way of achieving this? I'm happy to explore solutions, but as you say, from my perspective "this is just how Python packaging works" -- I never considered that adding this would cause any issues. Thanks, |
@AA-Turner sorry for the delay, I was in the process of vendoring several dozen more Python libraries at work, and wanted to see how pervasive this problem is before proposing any "solutions" here. Of the ~400 dependencies we (transitively) rely on, only Sphinx, PyTorch, and Airflow have this problem of breaking Bazel's underlying expectations that all dependencies should form a DAG. AFAICT, the only "Bazel-compliant" way to break this dependency cycle is to not allow $ pip install sphinxcontrib-XXX and get a completely-working package. One workaround that would work would be to define [project.optional-dependencies]
standalone = [
"sphinx > 5",
] then users that really wanted to just install a contrib package directly could manually do $ pip install sphinxcontrib-XXX[standalone] this would need to be paired with a runtime import check that prints a friendly error message if That said, this is a large amount of faff, when I think the Bazel team has basically decided that an upcoming version of Bazel will allow users to explicitly annotate groups of Given these considerations, I will close this Issue on the assumption(s) that
If my first assumption is wrong and you don't mind the solution I propose above, feel free to re-open and I can shoot over a PR that implements that approach. Thanks for your time! |
It's not just a "Bazel" thing. Any script that traces through dependencies trees will hit this. These commits broke the auto-generation of python packages at Ravenports. The workaround was to sed out the new Sphinx >= 5 lines out of the requirements before tracing the requirements. There are well over 400 python modules in Ravenports and the Sphinx family is the ONLY one that caused a circular dependency on itself. But yes, if you want to leave Sphinx as a dependency of htmlhelp and friends, then removing those as dependencies of Sphinx will resolve the circular dependency. |
|
Isn't possible to check the low Sphinx version boundary during runtime? For example directly in |
rules_python
Sphinx
and sphinxcontrib-*
It seems this causes more consternation than expected. @bruno-digitbio would you still be willing to implement the A |
fwiw, we are also hitting this (using bazel) |
this now appears to be totally broken (ie pinning deps no longer works) for 7.2.3+ as it depends upon a problem version of sphinxcontrib-serializinghtml |
@phlax as mentioned, I'm willing to adopt the [standalone] approach Bruno mentioned, if someone creates the PRs -- would you consider doing so, please? A |
@AA-Turner would be good to get reviews on ^^ |
Oh, sorry -- I had meant to reply. Please may you add the requires_sphinx calls mentioned above, and an entry to CHANGES (can be identical, just mentioning the dependency change). A |
Also, I believe |
im not seeing the ref
sure
sgtm |
ah ok - i see
|
Thank you! |
looking further - i had assumed there was some existing machinery that you wanted me to tap into but im not seeing any can you be more specific on what you are expecting here ? |
Each package should have a See: https://www.sphinx-doc.org/en/master/extdev/appapi.html#sphinx.application.Sphinx.require_sphinx A |
got it - not sure why my |
ive put date as |
I tend to use "unreleased" or "in development". |
k - if they are not being released today - then ill let you follow up to correct that |
@AA-Turner any chance of resolving this - we have to pin a load of deps in various build to workaround, and we cant upgrade |
@AA-Turner friendly bump. |
@AA-Turner friendly ping |
Thanks to @phlax for various pull requests, releases just out for:
The above packages no longer contain a dependency on A |
I'm still learning more of the details but am a bit puzzled by the approach to a fix here. My understanding is that Sphinx lists dependencies on various popular However, the dependency specifications from the Does anyone have context on why this problem had not been reported related to |
fwiw bazel now has a workaround for this which we are currently using (pending me removing it) regardless i think bazel is just flagging a genuine problem - ie circular dependencies - my understanding is that the landed solution is the best way to express this relationship not sure about historical without tracking through repos/versions but it started happening relatively recently |
Circular dependencies can be a problem sometimes, but I'm not yet convinced that they have been a problem for Sphinx. Sphinx lists a few In turn, some of those extensions depend on filtered versions of Sphinx, because they need features from the core of Sphinx at runtime to build documentation projects. An unresolvable circular dependency could be a problem, I agree - but this doesn't seem to have been the case here.
Can you provide any pointers for us (or me) to get started on that? Without knowing a specific reason for a disruptive change like this, I'd suggest we might want to revert or at least reconsider. Edit: attempt to clarify some wording, because the term 'build' can refer to building a Python package or using the installed packages to build a documentation project. |
I have to admit to not following this reference until a few moments ago:
So in fact there was a change in August 2023 to introduce the Sphinx dependency -- and that's likely what initiated the chain of events leading to Bazel's I'll do a bit more reading/investigation to confirm that -- and I do still think it's valid in theory for extensions to specify minimum supported Sphinx versions -- but clearly I hadn't understood enough of the details. |
Hi, just chiming into the pile to say that another package manager that was affected was Rez (a VFX-studio-oriented package and environment manager) which freaked out about having circular dependencies when at my employer we tried to package an internal Python project as a Rez package that was sourcing Sphinx. |
Describe the bug
I'm not sure if this is something that the Sphinx team even is interested in explicitly supporting, but the latest release of the
sphinxcontrib
tooling cannot be used withrules_python
(i.e. by users that use Bazel to install Sphinx).We build our monorepo's docs at work with Sphinx, and today when running some automated tooling to bump package versions, I encountered issues like the following:
These errors can easily be solved by pinning the older versions at one patch number below the current one:
The error is caused by the fact that at some point since the last release of these packages,
Sphinx>5
was added as an explicit dependency of these packages, creating the dependency loops demonstrated above. For example, compare thesetup.py
of serializainghtml at 1.1.5 to its newpyproject.toml
. This particular breakage appears to originate here.I recognize that most Python users are not also Bazel users, so I also created a minimal Git repo that reproduces the issue.
The
requirements_lock.txt
in this repo was produced by:$ conda create -n sphinx_repro python=3.10 $ conda activate sphinx_repro $ pip install sphinx $ pip freeze > requirements_lock.txt
Please let me know if there is interest in resolving this issue here, and if so if I can help in any way!
How to Reproduce
$ git clone git@github.com:brunobeltran/minimal-sphinx-repro.git $ cd minimal-sphinx-repro $ bazel build //...
Environment Information
Sphinx extensions
No response
Additional context
The text was updated successfully, but these errors were encountered: