Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: implement Starlark domain plugin for Sphinx #1909

Merged
merged 2 commits into from
May 23, 2024

Conversation

rickeylev
Copy link
Contributor

@rickeylev rickeylev commented May 20, 2024

This implements a Sphinx plugin to support Starlark as a first-class domain in Sphinx.
A Starlark domain allows writing object descriptions directly using Sphinx's object
markup language, which allows better integration with cross references, understanding
and distinguishing types, rendering information about things, and referencing types
from other projects.

Note that this doesn't affect the docs today because the proto_to_markdown tool is
still generating regular markdown; updating that to generate Sphinx domain markdown
will be done separately.

Summary of features:

  • Types and arguments can be documented using Python syntax, e.g. list[str] This includes
    unions, generics, and custom types. Each component of the type expression is linked and
    cross referenced appropriately. Each object can have its approprate pieces documented and
    defined (e.g. a rule can have attributes, attributes can have their defaults etc
    documented, etc)
  • An index of Starlark objects is generated automatically. This makes it easy, for example,
    to find the rules that have a particular attribute.
  • The following objects can be documented: functions, methods, rules, repository rules,
    providers, aspects, bzlmod extensions, tag classes, targets, flags, and
    attributes/fields of the aforementioned objects.
  • Arbitary docs can cross reference to Starlark types. e.g., a manually written "Getting
    Started" doc can write {bzl:obj}PyInfo.some_field and it will automatically link to
    the appropriate API docs.
  • Resolution of cross-references is much smarter and customizable. Instead of relying
    Markdown's link resolution rules, the Sphinx's crossreference resolution hooks are
    used. This allows more concise references (e.g., just a rule's base name), distinguishing
    a particular object type (e.g. a function vs rule), or referring to an absolute object.

This implements a Sphinx plugin to support Starlark as a first-class
domain in Sphinx. A Starlark domain allows writing object descriptions
directly using Sphinx's object markup language, which allows better
integration with cross references, understanding and distinguishing
types, rendering information about things, and referencing types
from other projects.

Note that this doesn't affect the docs today because the
proto_to_markdown tool is still generating regular markdown; updating
that to generate Sphinx domain markdown will be done separately.

Summary of features:
* Types and arguments can be documented using Python syntax, e.g.
  `list[str]` This includes unions, generics, and custom types. Each
  component of the type expression is linked and cross referenced
  appropriately. Each object can have its approprate pieces documented
  and defined (e.g. a rule can have attributes, attributes can have
  their defaults etc documented, etc)
* An index of Starlark objects is generated automatically. This makes
  it easy, for example, to find the rules that have a particular
  attribute.
* The following objects can be documented: functions, methods, rules,
  repository rules, providers, aspects, bzlmod extensions, tag classes,
  targets, flags, and attributes/fields of the aforementioned objects.
* Arbitary docs can cross reference to Starlark types. e.g., a manually
  written "Getting Started" doc can write `{bzl:obj}PyInfo.some_field` and
  it will automatically link to the appropriate API docs.
Copy link
Collaborator

@aignas aignas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The stardocs.py extension being a single file make the thing a little unwieldy imho. Having it across multiple could improve the readability, but it also has a lot of similar logic, so I am in favour of continuing with a single big file for now.

@@ -273,6 +278,7 @@ def _run_sphinx(ctx, format, source_path, inputs, output_prefix):
mnemonic = "SphinxBuildDocs",
progress_message = "Sphinx building {} for %{{label}}".format(format),
env = env,
execution_requirements = {"no-sandbox": ""},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why no-sandbox?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops, debugging stuff left over. Removed.


# Make a //foo:bar.bzl convert to foo.bar, not .foo.bar
if label.startswith("//"):
label = label.lstrip("/")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically you can have dots in the repo name, so this may break or have degenerate cases.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahhh, good point. Directories and files could have dots in them, too.

I've added a TODO for this for now. I think I'll change the internal representation to use @repo//file%symbol notation instead. The dotted notation grew out of originally only having within-same-file references, but it didn't evolve well once targets became cross-referencable, and I found it more natural to write the @repo//foo:bar%baz in docs rather than repo.foo.bar.baz.

@rickeylev rickeylev enabled auto-merge May 23, 2024 01:38
@rickeylev rickeylev added this pull request to the merge queue May 23, 2024
Merged via the queue into bazelbuild:main with commit c7defbc May 23, 2024
4 checks passed
@rickeylev rickeylev deleted the sphinx.stardoc.plugin branch May 23, 2024 03:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants