Use sphinx.ext.extlinks to add Sphinx roles for links to external package docs

[dimpase]

We wish to link to external packages' documentation:

• if the package documentation is installed locally in the Sage distribution, using the file link
• in downstream packaging, using a file link appropriate for the doc installation scheme of the system (without having to patch the documentation)
• if it is not available locally, using an http link to an online version of the package's documentation.

To this end, we use the sphinx.ext.extlinks facility, for which we already have infrastructure in sage.misc.sagedoc, assigning a Sphinx role to every external document.

We use a systematic naming scheme for these roles: :spkg_SPKG_DOC: where DOC is usually just doc but if a package has several manuals, it can be used to distinguish them.

In this ticket, we do this for the documents covered by a previous iteration of this ticket:

• :spkg_gc_doc: GC garbage collector <@SAGE_SHARE@/doc/gc/overview.html>_
• :spkg_giac_doc: Giac/Xcas documentation <@SAGE_SHARE@/doc/giac/index.html>_ https://www-fourier.ujf-grenoble.fr/~parisse/giac/doc/en/cascmd_en/cascmd_en.html
• :spkg_ppl_doc: PPL User manual <@SAGE_SHARE@/doc/ppl/ppl-user-1.2-html/index.html>_ https://www.bugseng.com/products/ppl/documentation/user/ppl-user-1.2-html/index.html
• :spkg_pplpy_doc: Python Wrapper for PPL (pplpy) <@SAGE_SHARE@/doc/pplpy/index.html>_
• :spkg_singular_doc: Singular User Manual <@SAGE_SHARE@/singular/html/index.htm>_ https://www.singular.uni-kl.de/Manual/4-2-0/index.htm
• spkg_maxima_doc: Introduction to Maxima <@SAGE_SHARE@/maxima/@MAXIMA_VERSION@/doc/html/intromax.html>_ https://maxima.sourceforge.io/docs/manual/maxima_0.html

The location of some of the locally installed documentation files can be determined at runtime, see #27495 comment:58 for maxima.
This should probably go through sage.feature.

Related:

• add intersphinx mapping for SciPy #29231 add intersphinx mapping for SciPy
• https://groups.google.com/g/sage-devel/c/oWkFwhinoMc/m/JpighcELBgAJ
• pip-installable packages sagemath-doc-src, sagemath-doc-inventory, sagemath-doc-html, sagemath-doc-pdf #29868 pip-installable packages sagemath-doc-src, sagemath-doc-inventory, sagemath-doc-html, sagemath-doc-pdf

CC: @tobihan @kcrisman @kiwifb @embray @haraldschilly @mkoeppe @antonio-rojas @mwageringel @videlec

Component: documentation

Keywords: sd111

Issue created by migration from https://trac.sagemath.org/ticket/27495

[dimpase]

Branch: u/dimpase/doc/docrefs

[dimpase]

Commit: f82a7c2

[dimpase]

Author: Dima Pasechnik

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

13c2fd5

trac 27495: links to external docs in reference index.rst

[sagetrac-git]

Changed commit from f82a7c2 to 13c2fd5

[dimpase]

comment:3

I don't know why it has taken so long to get started with this, to link together all these loose spkg docs in SAGE_LOCAL - something one might not even notice otherwise.

At least now SageNB docs are nicely linked - and available as HTML, in particular this will be there on the website, I suppose.

Let's get it in quickly, and add more on a followup ticket.

[kiwifb]

comment:4

Nice intentions I guess. From the point of view of sage in isolation it looks ok.

I don't use configure in sage-on-gentoo, so I would rather want something that applies from setup.py. Link to other packages documentation, all distros will probably want to patch. Note how it is all over the place? There is no way we let things be that messy. Documentation lives under some form of /usrshare/doc/, /usr/share/$PKG/doc and variations are frowned upon. But that's not easily fixable without you guys deciding on a standard and applying it across the board. We can wait a few more years before getting there.

[sagetrac-git]

Changed commit from 13c2fd5 to 85299a3

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

85299a3

linking pplpy docs

[dimpase]

comment:6

It seems to be a tall order to harmonise naming schemes and locations for docs of all the Sage packages.

I'd rather take what each package provides and in spkg-configure.m4 record this location into a variable (perhaps if the package is not built, but a system version used, then no doc link needs to be generated at all).

Another important reason for having this is that authors of external packages sometimes are annoyed by Sage incorporating their work silently. This showing/linking of docs will make it less silent...

[jhpalmieri]

comment:7

As fbissey said, what about doing this with Python instead of configure? Perhaps in src/sage_setup/docbuild/__init__.py we could modify index.rst, or take index.rst.in as input and write index.rst as output.

Or if it's just reference/index.rst that's relevant, it could be done in reference/conf.py.

[dimpase]

comment:8

Replying to @jhpalmieri:

As fbissey said, what about doing this with Python instead of configure?

Why? What's wrong with doing it at configuration time?
Should we redo the whole Sage building process with Python only? :-)

Mind you, Python does not have tools for this, autoconf does, as you see.

Perhaps in src/sage_setup/docbuild/__init__.py we could modify index.rst, or take index.rst.in as input and write index.rst as output.

Or if it's just reference/index.rst that's relevant, it could be done in reference/conf.py.

[kiwifb]

comment:9

Replying to @dimpase:

Replying to @jhpalmieri:

As fbissey said, what about doing this with Python instead of configure?

Why? What's wrong with doing it at configuration time?
Should we redo the whole Sage building process with Python only? :-)

That, you see, is the result of conflating "sage the distribution" with "sage the software". A confusion made easier by the fact that sage the software is not distributed as a proper package like the rest of its dependencies. The top configure is really mostly for sage the distribution and mingling the top level configure to stuff in sage the software means that you are not wanting sage the software to be pip installable ever.

Mind you, Python does not have tools for this, autoconf does, as you see.

Perhaps in src/sage_setup/docbuild/__init__.py we could modify index.rst, or take index.rst.in as input and write index.rst as output.

Or if it's just reference/index.rst that's relevant, it could be done in reference/conf.py.

Yes I am aware that what you are thinking of is easy with autotools but not trivial with python. But sometimes doing the easy stuff is just painting yourself in a corner in the long run.

[dimpase]

comment:10

We don't have a special manual for Sage the software (sagelib), only for Sage the distro. This ticket makes no attempt to change this situation, it merely improves the existing reference manual, in particular corrects the fact that SageNB docs are no longer available on http://doc.sagemath.org.

One would surely welcome an effort to partition the docs between sagelib and Sage the distro, but arguably a large part of the top index.rst, including everything on this ticket, would not belong to sagelib docs.

[timokau]

comment:12

Replying to @kiwifb:

Link to other packages documentation, all distros will probably want to patch. Note how it is all over the place? There is no way we let things be that messy. Documentation lives under some form of /usrshare/doc/, /usr/share/$PKG/doc and variations are frowned upon.

+1, could we make the individual doc locations configurable?

[dimpase]

comment:13

Replying to @timokau:

Replying to @kiwifb:

Link to other packages documentation, all distros will probably want to patch. Note how it is all over the place? There is no way we let things be that messy. Documentation lives under some form of /usrshare/doc/, /usr/share/$PKG/doc and variations are frowned upon.

+1, could we make the individual doc locations configurable?

individual packages have their own ideas where the docs should go. Mostly it is
either /usr/share/$PKG/doc or /usr/share/doc/$PKG, but not always. E.g. cmake puts docs in $prefix/doc. They also don't always have a topmost index.html, or quite often only provide pdf docs. Some probably have it configurable, some (most?) probably not.

Some packages have only little English documentation, e.g. giac.

This ticket is not meant to sort this huge mess out. It rather fixes most glaring omissions and provides a heads-up on docs that otherwise merely collect digital dust in $SAGE_LOCAL.

[timokau]

comment:14

Replying to @dimpase:

individual packages have their own ideas where the docs should go

I'm not talking about configuring where they are installed (which is a sage-the-distribution concern), but where sage looks for them when they are linked. Distributions don't install the docs of dependencies through sage's build system.

[dimpase]

comment:15

Replying to @timokau:

Replying to @dimpase:

individual packages have their own ideas where the docs should go

I'm not talking about configuring where they are installed (which is a sage-the-distribution concern), but where sage looks for them when they are linked. Distributions don't install the docs of dependencies through sage's build system.

I can give templates used in index.rst.in more generic names. Please suggest what would be suitable.

[timokau]

comment:16

Being able to override them with SAGENB_DOC, SINGULAR_DOC etc. would be helpful. I also currently don't use the ./configure script, but I don't see any reason I couldn't add that.

I agree that it is a step backwards from the "pip-installability" goal though.

[dimpase]

comment:17

Replying to @timokau:

Being able to override them with SAGENB_DOC, SINGULAR_DOC etc. would be helpful. I also currently don't use the ./configure script, but I don't see any reason I couldn't add that.

OK, will do.

I agree that it is a step backwards from the "pip-installability" goal though.

It is not really so, as the top level reference manual already has things that are not about sagelib, e.g. some of it are to be in the development manual.

I agree though that it's desirable to add one more level to Sage distro docs, something akin to
http://doc.sagemath.org/

[jhpalmieri]

comment:18

Replying to @dimpase:

Replying to @jhpalmieri:

As fbissey said, what about doing this with Python instead of configure?

Why? What's wrong with doing it at configuration time?
Should we redo the whole Sage building process with Python only? :-)

Mind you, Python does not have tools for this, autoconf does, as you see.

It's a search-and-replace; I think Python can handle it. Google led me to this, for example, for a generic search-and-replace:

import fileinput

for line in fileinput.FileInput(filename, inplace=True):
    print(line.replace(text_to_search, replacement_text))

[mkoeppe]

Changed reviewer from Dima Pasechnik to none

[mkoeppe]

Changed branch from u/jhpalmieri/doc/docrefs to none

[mkoeppe]

Changed commit from dab737d to none

[mkoeppe]

Changed author from Dima Pasechnik, John Palmieri to none

[mkoeppe]

comment:85

Setting new milestone based on a cursory review of ticket status, priority, and last modification date.

[embray]

comment:89

Just bikeshedding, but I don't know why the verbose names like :spkg_giac_doc: and not just something like :giac:. I understand giac_doc is supposed to refer to an SPKG specifically for the GIAC docs, but in the context of the documentation that should be a given anyways.

Also how does this relate to other Python packages that already use Sphinx, and can use intersphinx instead?

[mkoeppe]

comment:90

Replying to @embray:

Just bikeshedding, but I don't know why the verbose names like :spkg_giac_doc: and not just something like :giac:.

Fine with me to simplify these names.

Also how does this relate to other Python packages that already use Sphinx, and can use intersphinx instead?

The present ticket here is easier and can set the naming convention.

Packages where we can do intersphinx should use a compatible naming convention.

[mkoeppe]

comment:91

Setting a new milestone for this ticket based on a cursory review.