sage -docbuild: error building docs of single file

[DaveWitteMorris]

I tried a few different ways to build the docs of a single file, and got a few different errors. (But one of the methods works, so there is a workaround.) This is with sage 9.1.rc2 on Mac OS 10.15.3.

(1) My first attempt failed because 'DOT_SAGE' is not defined.

% sage --docbuild file=src/sage/plot/text.py pdf
Error building the documentation.
Traceback (most recent call last):
...
  File "/Users/dmorris/Documents/misc/Programs/sage3/local/lib/python3.7/site-packages/sage_setup/docbuild/__init__.py", line 1165, in __init__
    base_dir = os.path.join(DOT_SAGE, 'docbuild', module_name)
NameError: name 'DOT_SAGE' is not defined

(2) To avoid the need for DOT_SAGE, I gave an explicit build path. This worked better (it made pdf files of all of the plots), but failed with an AssertionError (and subsequent OSError): assert app.outdir.startswith(SAGE_DOC):

% sage --docbuild -o/Users/dmorris/tmp file=src/sage/plot/text.py pdf
...
[latex    ] Exception occurred:
[latex    ]   File "/Users/dmorris/Documents/misc/Programs/sage3/src/sage_setup/docbuild/ext/multidocs.py", line 220, in citation_dir
[latex    ]     assert app.outdir.startswith(SAGE_DOC)
...
OSError: Exception occurred:

(3) To eliminate the AssertionError, I added a print statement to tell me the value of SAGE_DOC, and I used it for the build path. This worked! (The release number and author show up as "unknown" in the pdf file, but I am guessing that this is the expected behavior.)

% sage --docbuild -o/Users/dmorris/Documents/misc/Programs/sage3/local/share/doc/sage file=src/sage/plot/text.py pdf
[latex    ] building [latex]: all documents
...
Output written on text.pdf (23 pages, 527224 bytes).
Transcript written on text.log.
=== TeX engine is 'pdfTeX'
Latexmk: Index file 'text.idx' was written
Latexmk: Log file says output to 'text.pdf'
Latexmk: All targets () are up-to-date
Build finished.  The built documents can be found in /Users/dmorris/Documents/misc/Programs/sage3/local/share/doc/sage/text/output/pdf

(4) To save myself some typing, I tried again with a local path for the build directory, but got

RuntimeError: failed to run $MAKE all-pdf in local/share/doc/sage/text/output/latex

because

mv: rename text.pdf to local/share/doc/sage/text/output/pdf: No such file or directory`.

The directory actually did get created (but is empty). However, it seems that the local path fails because the $MAKE command is not running in my working directory (where I ran the sage --docbuild command). I don't know whether this is a bug, or the expected behavior.

% sage --docbuild -o local/share/doc/sage file=src/sage/plot/text.py pdf
[latex    ] building [latex]: all documents
[latex    ] updating environment: 1 added, 0 changed, 0 removed
[latex    ] Merging environment/index files...
[latex    ] ... done (0 todos, 1 index, 0 citations, 0 modules)
[latex    ] WARNING: autodoc: failed to import module 'text'; the following exception was raised:
[latex    ] No module named 'text'
[latex    ] processing text.tex...index
[latex    ] resolving references...
[latex    ] writing... done
[latex    ] copying TeX support files...
[latex    ] done
[latex    ] The LaTeX files are in local/share/doc/sage/text/output/latex.
[latex    ] Run 'make' in that directory to run these through (pdf)latex
[latex    ] (use `make latexpdf' here to do that automatically).
LaTeX file written to local/share/doc/sage/text/output/latex
latexmk -pdf -dvi- -ps-  'text.tex'
Latexmk: This is Latexmk, John Collins, 26 Dec. 2019, version: 4.67.
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': The following rules & subrules became out-of-date:
      'pdflatex'
------------
Run number 1 of rule 'pdflatex'
------------
------------
Running 'pdflatex   -recorder  "text.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (preloaded format=pdflatex)
...
Transcript written in text.ilg.
Latexmk: applying rule 'pdflatex'...
Rule 'pdflatex': File changes, etc:
   Changed files, or newly in use since previous run(s):
      'text.aux'
      'text.ind'
      'text.out'
------------
Run number 2 of rule 'pdflatex'
------------
------------
Running 'pdflatex   -recorder  "text.tex"'
------------
This is pdfTeX, Version 3.14159265-2.6-1.40.21 (TeX Live 2020) (preloaded format=pdflatex)
...
Output written on text.pdf (4 pages, 7317 bytes).
Transcript written on text.log.
=== TeX engine is 'pdfTeX'
Latexmk: Index file 'text.idx' was written
Latexmk: Log file says output to 'text.pdf'
Latexmk: All targets () are up-to-date
mv: rename text.pdf to local/share/doc/sage/text/output/pdf: No such file or directory
Error building the documentation.
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/Users/dmorris/Documents/misc/Programs/sage3/local/lib/python3.7/site-packages/sage_setup/docbuild/__main__.py", line 2, in <module>
    main()
  File "/Users/dmorris/Documents/misc/Programs/sage3/local/lib/python3.7/site-packages/sage_setup/docbuild/__init__.py", line 1720, in main
    builder()
  File "/Users/dmorris/Documents/misc/Programs/sage3/local/lib/python3.7/site-packages/sage_setup/docbuild/__init__.py", line 252, in pdf
    raise RuntimeError(error_message%(command, tex_dir))
RuntimeError: failed to run $MAKE all-pdf in local/share/doc/sage/text/output/latex

Component: build

Author: John Palmieri

Branch/Commit: b0310af

Reviewer: Matthias Koeppe

Issue created by migration from https://trac.sagemath.org/ticket/29651

[jhpalmieri]

comment:1

This fixes the problem for me. The only part I'm not happy about is silencing the warning in sphinxbuild.py, because that message could indicate a problem in some circumstances:

diff --git a/src/sage_setup/docbuild/__init__.py b/src/sage_setup/docbuild/__init__.py
index 128117b960..6c4761859b 100644
--- a/src/sage_setup/docbuild/__init__.py
+++ b/src/sage_setup/docbuild/__init__.py
@@ -59,7 +59,7 @@ import sphinx.ext.intersphinx
 import sage.all
 from sage.misc.cachefunc import cached_method
 from sage.misc.misc import sage_makedirs
-from sage.env import SAGE_DOC_SRC, SAGE_DOC, SAGE_SRC
+from sage.env import SAGE_DOC_SRC, SAGE_DOC, SAGE_SRC, DOT_SAGE
 
 from .build_options import (LANGUAGES, SPHINXOPTS, PAPER, OMIT,
      PAPEROPTS, ALLSPHINXOPTS, NUM_THREADS, WEBSITESPHINXOPTS,
@@ -1185,6 +1185,9 @@ html_title = project
 html_short_title = project
 htmlhelp_basename = name
 
+extensions.remove('multidocs')
+extensions.remove('inventory_builder')
+
 latex_domain_indices = False
 latex_documents = [
   ('index', name + '.tex', u'Documentation for {}',
diff --git a/src/sage_setup/docbuild/sphinxbuild.py b/src/sage_setup/docbuild/sphinxbuild.py
index fe7eba43b2..31fdaf4bfb 100644
--- a/src/sage_setup/docbuild/sphinxbuild.py
+++ b/src/sage_setup/docbuild/sphinxbuild.py
@@ -110,6 +110,7 @@ class SageSphinxLogger(object):
             re.compile('WARNING: Any IDs not assiend for figure node'),
             re.compile('WARNING: .* is not referenced'),
             re.compile('WARNING: Build finished'),
+            re.compile('WARNING: unknown config value \'multidoc_first_pass\''),
             )
         self._useless_chatter += self._ignored_warnings
 

[DaveWitteMorris]

comment:2

Thanks! This does make (1) and (2) work for me (and (3) was already working). Do you want to make the patches into a branch (or should I)? I don't understand the docbuild system, so I don't have a suggestion for an alternative to silencing the warning (and I can't review this fix), but maybe I will before 9.2 comes out.

Method (4) still doesn't work, but I don't think it really needs to be fixed. However, unless there is already some reason that a user shouldn't expect relative paths to work for the build directory, then I would suggest mentioning the need for absolute paths in the documentation (and in sage --docbuild -h). That's a patch that I could probably do.

[jhpalmieri]

comment:3

Let me think more about the change to sphinxbuild.py. There should be another way to do this.

[jhpalmieri]

comment:5

This branch partially works for me:

• it works with ./sage --docbuild file=src/sage/plot/text.py html, same with pdf instead of html.
• it does not work with ./sage --docbuild file=src/sage/algebras/steenrod/steenrod_algebra.py html, because that file has citations to references in src/doc/en/reference/references/index.rst.
• it does not work with a file outside of the Sage library unless that file is in Python's path (for example in the current directory): "WARNING: autodoc: failed to import module 'foo'" . The documentation is ambiguous; should it work for such files?

[jhpalmieri]

Branch: u/jhpalmieri/single-file-docbuild

[jhpalmieri]

Commit: 4e167e9

[jhpalmieri]

comment:7

I'm marking this as "needs review", but it could perhaps use improvements, as indicated in comment:5.

---

New commits:

4e167e9

trac 29651: docbuild a single file

[mkoeppe]

comment:8

$ ./sage -docbuild file=src/sage/geometry/polyhedron/base.py html 
[html     ] building [html]: targets for 1 source files that are out of date
[html     ] updating environment: [new config] 1 added, 0 changed, 0 removed
[html     ] WARNING: autodoc: failed to import module 'base'; the following exception was raised:
[html     ] attempted relative import with no known parent package
[html     ] writing additional pages...  searchdone
[html     ] copying static files... ... done
[html     ] dumping search index in English (code: en)... done
[html     ] The HTML pages are in ../../../../.sage/docbuild/base/output/html.
Error building the documentation.

[mkoeppe]

comment:9

Note "attempted relative import with no known parent package"

[mkoeppe]

comment:10

I can confirm that ./sage --docbuild file=src/sage/plot/text.py html works

[jhpalmieri]

comment:11

Replying to @mkoeppe:

Note "attempted relative import with no known parent package"

I don't know what that means.

[sagetrac-git]

Changed commit from 4e167e9 to b0310af

[sagetrac-git]

Branch pushed to git repo; I updated commit sha1. New commits:

b0310af

trac 29651: fix module names with underscores

[jhpalmieri]

comment:13

Maybe the module name needs to be set more intelligently if the path starts with SAGE_SRC, or something like that.

[mkoeppe]

comment:14

Looks like the approach of SingleFileBuilder, creating a symlink to the source file, is not compatible with from .module import ...

[mkoeppe]

comment:15

Replying to @jhpalmieri:

Maybe the module name needs to be set more intelligently if the path starts with SAGE_SRC, or something like that.

I just wrote something like this for #30452

[mkoeppe]

Author: John Palmieri

[mkoeppe]

Reviewer: Matthias Koeppe

[mkoeppe]

comment:17

In any case, this is already an improvement. We can improve it further on a follow-up ticket.

[mkoeppe]

comment:18

Replying to @mkoeppe:

We can improve it further on a follow-up ticket.

I have created #30475 for this.

[vbraun]

Changed branch from u/jhpalmieri/single-file-docbuild to b0310af