Lean on Typedoc instead of maintaining Docgen

[adamziel]

Is there any real upside to maintaining our own documentation tool? Docgen confuses us with these errors demanding a hotfix, e.g. Here:

• Docgen: Fix issue where function token can't be found #49812
• [TypeScript] Preserve the generic signature of getEntityRecord and getEntityRecords through currying #44453
• docgen: Unwrap wrapped selectors when inferring types of JSDoc params #40236

TypeScript syntax keeps evolving, and even the existing one isn’t exhaustively described by Docgen.

Have we ever considered switching to another tool? For example https://typedoc.org/ which @gziolo once recommended to me. I moved WordPress Playground from a custom documenter to TypeDoc and the results are splendid. It works like a charm even for the weirdest TS corner-cases and when it fails, it fails gracefully and still produces some output instead of an error.

Github search doesn't yield too many results. @youknowriad or @gziolo – can you remember any discussions?

also cc @tyxla @jsnajdr @noahtallen @noisysocks @t-hamano @justintadlock @sarayourfriend @aaronrobertshaw @mirka

[youknowriad]

I was not too much involved in the initial work on docgen but I think the main issue is that at the time there were no third party options that allowed us to achieve the results we wanted. Also let's make sure that which ever solution we pick is not typescript specific but works for both JS and typescript.

[sarayourfriend]

Also let's make sure that which ever solution we pick is not typescript specific but works for both JS and typescript.

I think this is the primary issue with TypeDoc, or at least was when I last looked at it. docgen extracts documentation from JSDoc or TypeScript annotations statically, without compiling or running the code. TypeDoc relies on the TypeScript compiler to provide it the type information (IIRC) and therefore does nothing with regular JSDoc comments.

[johnhooks]

Is the concern, many of the current JSDoc comments were not written with type checking in mind, and an attempt to run the TypeScript compiler will generate type errors?

There seems to be a large effort to add type checking, whether with JSDoc comments or strait TS in some instances. Is using a tool that requires compatibility with tsc a deal breaker?

[sarayourfriend]

Is the concern, many of the current JSDoc comments were not written with type checking in mind, and an attempt to run the TypeScript compiler will generate type errors?

Yes, in fact you can guarantee that JSDoc comments will not pass type checking.

@dmsnell and others are working on adding type annotations to Gutenberg, but it is a slow process. Gutenberg is a complicated, dynamic application that was not written with typing in mind. If TypeDoc can extract non-type-checked JSDoc, then maybe it's more realistic an alternative. If those JSDoc comments need to be included in the TypeScript project and pass checking, then it's probably less workable. Using implicit any with specific spot-fixes might do the trick, I think Dennis has thought about that a bit as well.

[johnhooks]

@adamziel I would be very willing to support this effort. I did a quick look into TSDoc JSDoc support and it looks like it can do it, which makes sense to me. TypeScript processes JSDocs so it should be possible.

JSDoc is the de-facto "standard" for comments, but does not specify a rigorous grammar and is fully implemented only by the official JSDoc tool. TypeDoc aims to recognize most JSDoc comments in a manner similar to how they are handled by TypeScript and Visual Studio Code.

There might need to be a refactor of the JSDoc comments for compatibility.

[johnhooks]

Update: I just tried tsdoc on my work in #48604. It created a beautiful interlinked documentation site from the JSDocs type comments.

[adamziel]

@johnhooks would that work for the entire Gutenberg repo? If so, a proof of concept PR would go a long way to move this discussion forward.

[johnhooks]

@adamziel it should work for each package that has a tsconfig and can built its types without errors. I will attempt doing it for the entire repo and report back.

[adamziel]

In case that's useful, this is the config I use in WordPress Playground:

https://github.com/WordPress/wordpress-playground/blob/trunk/typedoc.js

It requires adding the following typedoc section to package-specific package.json files:

https://github.com/WordPress/wordpress-playground/blob/64c3e8420912e2ceeae62941eba53e3a149c4ddb/packages/php-wasm/node/package.json#L18-L23

[johnhooks]

Related to #48998, and interest in an API reference for Gutenberg that is similar to the WordPress Code Reference.