Sphinx extlinks for ecl, gap, giac, maxima, meson, ppl, qepcad, scip, singular, soplex #37598
Conversation
mkoeppe
changed the title
mkoeppe
changed the title
mkoeppe
changed the title
…:gap:, :gap_package: roles
…:ecl:, :common_lisp:, :qepcad: roles
…:meson:, :polymake:, :scip:, :soplex: roles
mkoeppe
force-pushed
the
extlinks_upstream
branch
from
bbab6a6 to
5250624
Compare
src/sage/misc/sagedoc.py
Outdated
| 'common_lisp': ('https://www.lispworks.com/documentation/lw50/CLHS/Body/%s.htm', 'Common Lisp: %s'), | ||
| 'ecl': ('https://ecl.common-lisp.dev/static/manual/%s.html', 'ECL: %s'), | ||
| 'gap': ('https://gap-system.org/Manuals/doc/ref/%s_mj.html', 'GAP: %s'), | ||
| 'gap_package': ('https://www.gap-system.org/Packages/%s.html', 'GAP package %s'), |
There was a problem hiding this comment.
Just as heads-up, those URLs are probably not going to exist for much longer. Not sure what the goal here is, is it to link to the homepages of these packages? Or their documentation? Or what?
There was a problem hiding this comment.
Anything really, I don't need anything very specific here for now. Package documentation would be fine.
There was a problem hiding this comment.
@fingolfin What URL would you recommend to use for the package documentation?
There was a problem hiding this comment.
@fingolfin And more generally (not for this PR) -- what would be the best way to link to specific functions or other objects in the GAP documentation in a stable way?
(For packages that use Sphinx to build their documentation, we use the cross-referencing tool Intersphinx for this purpose.)
There was a problem hiding this comment.
While we still mirror the docs of distributed GAP packages on www.gap-system.org, the URLs there are not quite uniform, as the entry pages have different names / paths for each package.
This information is contained in the metadata of each package. That metadata is available in a JSON file (e.g. https://github.com/gap-system/gap/releases/download/v4.13.0/package-infos.json.gz for the packages distributed with GAP 4.13.0). I can provide further details if desired.
On the website we only link to the manual for the latest GAP release, and the manuals of the packages within that. So these are not really "stable".
I hope that we can eventually enhance this by adding version pickers. In preparation for this future (which may never come sigh but I am hoping), I made sure that from a certain point on, we had separate URLs for each GAP version -- these are not currently used anywhere! But as I wrote, I hope this will change.
The "regular" URLs for our mirrors of package manuals look like these (note the different paths after the package name)
- https://docs.gap-system.org/pkg/smallsemi/doc/chap0.html
- https://docs.gap-system.org/pkg/transgrp/htm/chapters.htm
The "versioned" ones look like this:
- https://docs.gap-system.org/v4.13.0/pkg/smallsemi/doc/chap0.html
- https://docs.gap-system.org/v4.13.0/pkg/transgrp/htm/chapters.htm
(So they are versioned by GAP version, not by package version. We could debate this at length, but let's not).
Anyway, so my suggestion would be to use these URLs.
But the annoying bit remains that you can't know the full URLs (.../doc/chap0.html or .../doc/chap0_mj.html or ..../htm/chapters.htm or maybe something else) without consulting that JSON file.
But perhaps we could just add a step to the script which updates the manuals on the GAP website for a new release, and add redirects ...
- from https://docs.gap-system.org/v4.13.0/pkg/smallsemi/ to https://docs.gap-system.org/v4.13.0/pkg/smallsemi/doc/chap0.html,
- from https://docs.gap-system.org/v4.13.0/pkg/transgrp/ to https://docs.gap-system.org/v4.13.0/pkg/transgrp/htm/chapters.htm,
- and so on.
Since that script already is parsing the JSON data and more, it should be fairly easy to do. We use Apache, so the script could just produce a .htaccess file with suitable rules... (Actually it might be even easier to write a separate script which consumes a package-infos.json and spits out the .htaccess file; then it'd be very easy to apply it retroactively to prior GAP releases).
All that said, I am very busy with other stuff right now so I don't want to promise anything. If someone would like to help out, I can provide further details.
There was a problem hiding this comment.
@fingolfin For example, might there be interest on your side in extending GAPDoc so it can output RST files and then go through Sphinx for the HTML build?
There was a problem hiding this comment.
Those anchors are hashes and are essentially permanent. For our website we basically keep an index file (see https://github.com/gap-system/GapWWW/blob/master/_data/help.json which is exported from GAP via this script.)
Regarding "output RST files and then go through Sphinx for the HTML build": I am not really familiar with RST files and Sphinx. I am pretty sure I'd have a hard time convincing most GAP package maintainers to learn about those and installing relevant parts. But I guess it might still be potentially useful as an optional backend with added features (search, the ability to keep multiple versions and switch between them come to mind) that perhaps then is only used by the GAP website (and possibly via GH Action jobs also on package websites, if we can provide package authors with a "drop in and forget" solution).
As such, I would find that potentially interesting. To be fully useful, also the GAP help system would have to be taught to deal with the resulting HTML (GAP can be set up so that the REPL help opens the HTML docs at the correct position). I guess this is less important if it only is about producing online versions of those docs.
I have zero interest in working on that, but if someone else did, I'd be happy to try it out and give feedback.
I have no idea if there is any chance to get such a patch into GAPDoc, though (the maintainer sometimes has... strong oppinions). But it might not even be necessary, GAPDoc already has three "backends" ("plain text with escape sequences" for the REPL help; HTML; PDF-via-latex), and I think this is done in a way that makes it easy to add extra backends. So such a thing could perhaps be done via an extension package.
There was a problem hiding this comment.
From the GAPDoc:
# - add hash strings of the latter for links in HTML and PDF version
# (such that links remain stable as long as the (sub)section exists
# and stays in the same chapter)
There was a problem hiding this comment.
I guess it might still be potentially useful as an optional backend with added features (search, the ability to keep multiple versions and switch between them come to mind) that perhaps then is only used by the GAP website (and possibly via GH Action jobs also on package websites, if we can provide package authors with a "drop in and forget" solution).
Yes, that's what I would imagine. No need to bring local building by GAP users/developers into the equation.
I have zero interest in working on that
Likewise, but that's a kind of project that would pretty easy to hire someone for. No math required!
There was a problem hiding this comment.
Ah, I wish I could just hire someone for that kind of work, but in general this is very difficult with German university funding and general regulations on hiring people :/. Essentially I can hire local students (but good luck finding one with the skills). sigh
Co-authored-by: Max Horn <max@quendi.de>
|
looks good to me except that |
|
The link https://www.usna.edu/CS/qepcadweb/B/user/EnterForm.html works just fine here. |
|
Even OK then. |
|
Thank you! |
|
Documentation preview for this PR (built with commit 13e15e9; changes) is ready! 🎉 |
…flint, fpylll, gmpy2, ipywidgets, matplotlib, mpmath, networkx, numpy, rpy2, scipy, sympy
<!-- ^ Please provide a concise and informative title. -->
<!-- ^ Don't put issue numbers in the title, do this in the PR
description below. -->
<!-- ^ For example, instead of "Fixes sagemath#12345" use "Introduce new method
to calculate 1 + 2". -->
<!-- v Describe your changes below in detail. -->
<!-- v Why is this change required? What problem does it solve? -->
<!-- v If this PR resolves an open issue, please link to it here. For
example, "Fixes sagemath#12345". -->
[Intersphinx](https://www.sphinx-
doc.org/en/master/usage/extensions/intersphinx.html) allows us to refer
to functions and classes in other projects using the standard Sphinx
roles `:func:`, `:class:`, ...
Examples in the documentation preview:
- [References to FLINT functions in
sage.libs.flint.arith_sage](https://deploy-preview-37575--sagemath.netli
fy.app/html/en/reference/libs/sage/libs/flint/arith_sage#sage.libs.flint
.arith_sage.bell_number)
- [References to SciPy functions in
sage.manifolds.differentiable.integrated_curve](https://deploy-
preview-37575--sagemath.netlify.app/html/en/reference/manifolds/sage/man
ifolds/differentiable/integrated_curve#sage.manifolds.differentiable.int
egrated_curve.IntegratedCurve.solve)
- [References to NetworkX functions in
sage.graphs.generic_graph](https://deploy-preview-37575--sagemath.netlif
y.app/html/en/reference/graphs/sage/graphs/generic_graph#sage.graphs.gen
eric_graph.GenericGraph.export_to_file)
- [Examples in the Developer Guide](https://deploy-preview-37575--
sagemath.netlify.app/html/en/developer/sage_manuals#hyperlinks)
Based on a rebased version of sagemath#29231 by @mwageringel. In addition to the
`scipy` intersphinx done in sagemath#29231, here we add a number of relevant
Python libraries, as well as `flint`, whose docs are built with Sphinx
as well.
To address concerns in the discussion there about the docbuild
contacting remote servers for the inventory files, here we instead
vendor the inventory files:
```
$ ls -l src/doc/common/_vendor
-rw-r--r-- 1 mkoeppe staff 1942 Mar 9 21:20 cvxopt.inv
-rw-r--r-- 1 mkoeppe staff 13251 Mar 9 21:20 cvxpy.inv
-rw-r--r-- 1 mkoeppe staff 10728 Mar 9 21:20 cypari2.inv
-rw-r--r-- 1 mkoeppe staff 775 Mar 9 21:20 cysignals.inv
-rw-r--r-- 1 mkoeppe staff 246266 Mar 9 21:20 flint.inv
-rw-r--r-- 1 mkoeppe staff 1603 Mar 9 21:20 fpylll.inv
-rw-r--r-- 1 mkoeppe staff 2891 Mar 9 21:20 gmpy2.inv
-rw-r--r-- 1 mkoeppe staff 10934 Mar 9 21:20 ipywidgets.inv
-rw-r--r-- 1 mkoeppe staff 105887 Mar 9 21:20 matplotlib.inv
-rw-r--r-- 1 mkoeppe staff 3115 Mar 9 21:20 mpmath.inv
-rw-r--r-- 1 mkoeppe staff 51830 Mar 9 21:20 networkx.inv
-rw-r--r-- 1 mkoeppe staff 78006 Mar 9 21:20 numpy.inv
-rw-r--r-- 1 mkoeppe staff 1449 Mar 9 21:20 pplpy.inv
-rw-r--r-- 1 mkoeppe staff 136166 Mar 9 21:20 python.inv
-rw-r--r-- 1 mkoeppe staff 3289 Mar 9 21:20 rpy2.inv
-rw-r--r-- 1 mkoeppe staff 112691 Mar 9 21:20 scipy.inv
-rw-r--r-- 1 mkoeppe staff 55596 Mar 9 21:20 sympy.inv
```
This extends our existing practice with the Python inventory (moved here
from its previous location `src/doc/common/python3.inv`). The new
command `sage -python -m sage_docbuild.vendor` retrieves the latest
versions of these files. (Distribution notes: These files are not
installed, as they are only needed at the build time of the
documentation. After sagemath#36730, they are part of the sdist of sagemath-doc-
html, but not part of the wheel.)
By setting `SAGE_DOC_REMOTE_INVENTORIES=yes`, this can be overridden, as
previously suggested in
sagemath#29231 (comment).
In this case it is first tried to contact the remote servers.
Fixes sagemath#29231 (stalled since 2020).
Fixes sagemath#27164 (stalled since 2019).
**Outside the scope of this PR:**
- adding such Intersphinx links everywhere (that would be a long-term
writing project - https://groups.google.com/g/sage-
devel/c/PfYpuOWt8xQ/m/Emn7pZd5AQAJ)
- linking to documentation of non-Sphinx projects (see sagemath#37598, sagemath#37577)
- any considerations how these .inv files could/should/would be taken
from our installations of these packages, or from system packages that
ship the documentation.
### 📝 Checklist
<!-- Put an `x` in all the boxes that apply. -->
- [x] The title is concise and informative.
- [ ] The description explains in detail what this PR is about.
- [x] I have linked a relevant issue or discussion.
- [ ] I have created tests covering the changes.
- [ ] I have updated the documentation accordingly.
### ⌛ Dependencies
<!-- List all open PRs that this PR logically depends on. For example,
-->
<!-- - sagemath#12345: short description why this is a dependency -->
<!-- - sagemath#34567: ... -->
URL: sagemath#37575
Reported by: Matthias Köppe
Reviewer(s): David Coudert, Matthias Köppe
|
I think this is making several RST lint checks fail |
|
I made an issue: #37711 -- Oh is this an issue because the linter is being run before the new docs have been built? Maybe patience is the solution here. |
|
this requires to add these also in |
We create a standard way to refer to the online manuals of these packages, which are generated by tools other than Sphinx and therefore cannot be linked to using Intersphinx.
This extends what has been already done with Pari.
Examples:
Fixes #27495.
Outside of the scope of this PR:
📝 Checklist
⌛ Dependencies