Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

docs: generate Starlark domain markup instead of regular markdown #1919

Merged
merged 6 commits into from
May 24, 2024

Conversation

rickeylev
Copy link
Contributor

This switches our doc generation over to using the Starlark domain markup from the sphinx_stardoc plugin instead of using regular markdown. This allows the docs generated from code to better integrate with each other and other parts of the doc site.

Overview of changes:

  • Makes the doc paths under the API directory more directly mirror their actual location. e.g. moves "defs.md" -> "python/defs.md". This is so the //tools doc entries have a more natural location, but can also be used for our other top-level directories.
  • Adds API docs for some of the well known targets we have. These aren't automatically generated, but use the Starlark domain markup, so integrate nicely with everything.
  • Ensures default values are parsable as Python expressions. Stardoc returns values like "" or 'Label(*, "//bar")' in some cases for the default value of args/attrs.
  • Ensures function signatures don't crash doc rendering. Stardoc gives bad/incomplete information, so reconstructing the original signature of a function is tricky.
  • Allows references flags using leading slashes and a value, e.g. --foo=bar. This makes it more natural to write while cross referencing to the flag.
  • Implements {any} xref resolution. It was just totally broken before.
  • Adds some additional bzl files that get documented.
  • Adds some more Bazel external references.
  • Fixes some missing bzl_library dependencies.
  • A few minor QoL improvements to the docs dev server:
    • Print the serving directory when CTRL+C is received. This makes it easier to find the raw files that are being generated.
    • Fix an error during shutdown about an unterminated generator.
  • The sphinx_stardocs.footer arg is removed. This was always just a hack to get extra link targets into the generated bzl docs. It's no longer needed when the bzl domain is used.
  • Using @repo//pkg:file.bzl%Name syntax is supported in type expressions (e.g. :type: option or {type} role) by quoting the label. The quoting is necessary because, under the hood, the expressions are parsed as Python.
  • Objects directives support an :origin-key directive. This records the label identity that Bazel sees for an object (as from the Stardoc origin_key field). The markdown generate doesn't generate this for everything yet because some things are documented twice (e.g. py_binary in defs.bzl and py_binary.bzl), which would cause a crash (type things trying to define the same id).
  • Add * and ** to var-args and var-kwargs in signatures.
  • Allow providers to be refered to using the type role. This allows providers to be referenced in :type: directives (e.g. in a provider field).

This switches our doc generation over to using the Starlark domain
markup from the sphinx_stardoc plugin instead of using regular
markdown. This allows the docs generated from code to better
integrate with each other and other parts of the doc site.

Overview of changes:

* Makes the doc paths under the API directory more directly mirror
  their actual location. e.g. moves "defs.md" -> "python/defs.md".
  This is so the //tools doc  entries have a more natural location,
  but can also be used for our other top-level directories.
* Adds API docs for some of the well known targets we have.
  These aren't automatically generated, but use the Starlark
  domain markup, so integrate nicely with everything.
* Ensures default values are parsable as Python expressions. Stardoc
  returns values like "<function foo>" or 'Label(*, "//bar")' in some
  cases for the default value of args/attrs.
* Ensures function signatures don't crash doc rendering. Stardoc
  gives bad/incomplete information, so reconstructing the original
  signature of a function is tricky.
* Allows references flags using leading slashes and a value,
  e.g. `--foo=bar`. This makes it more natural to write while
  cross referencing to the flag.
* Implements `{any}` xref resolution. It was just totally broken before.
* Adds some additional bzl files that get documented.
* Adds some more Bazel external references.
* Fixes some missing bzl_library dependencies.
* A few minor QoL improvements to the docs dev server:
  * Print the serving directory when CTRL+C is received. This makes it
    easier to find the raw files that are being generated.
  * Fix an error during shutdown about an unterminated generator.
* The `sphinx_stardocs.footer` arg is removed. This was always just
  a hack to get extra link targets into the generated bzl docs. It's
  no longer needed when the bzl domain is used.
* Using `@repo//pkg:file.bzl%Name` syntax is supported in type expressions
  (e.g. `:type:` option or `{type}` role) by quoting the label. The
  quoting is necessary because, under the hood, the expressions are
  parsed as Python.
* Objects directives support an `:origin-key` directive. This records the
  label identity that Bazel sees for an object (as from the Stardoc
  origin_key field). The markdown generate doesn't generate this for
  everything yet because some things are documented twice (e.g.
  py_binary in defs.bzl and py_binary.bzl), which would cause a
  crash (type things trying to define the same id).
* Add `*` and `**` to var-args and var-kwargs in signatures.
* Allow providers to be refered to using the `type` role. This allows
  providers to be referenced in `:type:` directives (e.g. in
  a provider field).
@rickeylev rickeylev requested a review from aignas May 23, 2024 22:03
Copy link
Collaborator

@aignas aignas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The docs look much better, thanks!

docs/sphinx/BUILD.bazel Show resolved Hide resolved
sphinxdocs/src/sphinx_stardoc/stardoc.py Show resolved Hide resolved
@rickeylev rickeylev enabled auto-merge May 24, 2024 03:42
@rickeylev rickeylev added this pull request to the merge queue May 24, 2024
Merged via the queue into bazelbuild:main with commit 6ca2f58 May 24, 2024
4 checks passed
@rickeylev rickeylev deleted the use.stardoc.domain branch May 24, 2024 15:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants