Autogenerate CRD docs #2116

super-harsh · 2022-02-25T01:10:15Z

What this PR does / why we need it:

This PR adds support to generate API docs for ASOv2 CRDs for the end-users. API docs will help the end-users to get a better understanding of the fields and values for CRDs.

Special notes for your reviewer:

This PR includes the generated API docs, task and script to generate the API docs. Docs are generated separately for each group and version. The tool we're using to generate the docs is gen-crd-api-reference-docs.

How does this PR make you feel:

If applicable:

this PR contains documentation

…les for CRDs

…preview.md

…-service-operator into feature/gen-api-docs

…in text html' hugo error

codecov-commenter · 2022-02-27T23:17:25Z

Codecov Report

Merging #2116 (e9fb5ef) into main (270280c) will decrease coverage by 0.00%.
The diff coverage is 11.11%.

@@            Coverage Diff             @@
##             main    #2116      +/-   ##
==========================================
- Coverage   56.99%   56.98%   -0.01%     
==========================================
  Files         585      585              
  Lines      106379   106387       +8     
==========================================
  Hits        60629    60629              
- Misses      38276    38284       +8     
  Partials     7474     7474

Impacted Files	Coverage Δ
.../generator/internal/astmodel/package_definition.go	`7.33% <0.00%> (-0.59%)`	⬇️
...2/tools/generator/internal/config/configuration.go	`19.87% <ø> (ø)`
...internal/codegen/pipeline/export_generated_code.go	`6.25% <25.00%> (ø)`
...tools/generator/internal/codegen/code_generator.go	`90.44% <100.00%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 270280c...e9fb5ef. Read the comment docs.

docs/hugo/config.toml

docs/api/template/config.json

scripts/gen-api-docs.sh

matthchr · 2022-03-02T01:05:06Z

v2/tools/generator/internal/astmodel/package_definition.go

 	buf := &bytes.Buffer{}
-	err := groupVersionFileTemplate.Execute(buf, pkgDef)
+	defer buf.Reset()


What's the point of doing this given that buf goes out of scope when the function ends?

It will dealloc the resources and garbage collection efficiency will be better.

Really? If you look at the Reset() implementation:

// Reset resets the buffer to be empty, // but it retains the underlying storage for use by future writes. // Reset is the same as Truncate(0). func (b *Buffer) Reset() { b.buf = b.buf[:0] b.off = 0 b.lastRead = opInvalid }

You can see there's no actual space savings here: https://go.dev/play/p/zXGZkI8C3Rv. Even if this did actually deallocate space, I don't think there's anything that makes a difference between waiting til the GC comes to clean up the slice whose space you deallocated, vs waiting for the GC to come clean up the buf itself. AFAIK both will take the same amount of time. @Porges might know something I don't there though.

v2/tools/generator/internal/astmodel/package_definition.go

matthchr · 2022-03-02T01:09:21Z

scripts/gen-api-docs.sh

+    GROUPNAME=$(basename $(dirname $package))
+
+    #Filter the main CRD packages
+    if [[ $PACKAGE_VERSION =~ $PATTERN ]] && [[ "$PACKAGE_VERSION" != *"storage" ]] 


Is this actually needed now given the storage packages also do not have doc.go? (Not entirely sure what doc.go gets the tool so just checking)

This checks saves us time in generating docs, if the tool goes through the storage packages then it tries to iterate through all the elements inside the package and then come up with an error no API packages found in ./api/batch/v1alpha1api20210101storage which takes a few(5-10) seconds.

Maybe leave a comment explaining that here?

docs/api/template/config.json

…arameter in scripts/gen-api-docs.sh; Add gen-crd-api-reference-docs installation inside .devcontainer/install-dependencies.sh; Add genruntime external reference for gen-crd-api-reference-docs in template/config.json

matthchr

This looks good to me. There are still some rough edges, for example the fact we generate documentation for the ARM types. It might be worth filing an item to stop doing that (we should just document the k8s types), but I think this is a good starting point so going ahead and approving.

matthchr · 2022-03-04T21:18:43Z

.devcontainer/install-dependencies.sh

@@ -62,6 +62,9 @@ go get \
 go install -tags extended github.com/gohugoio/hugo@v0.88.1
 go install github.com/wjdp/htmltest@v0.15.0

+# for api docs 
+go install github.com/ahmetb/gen-crd-api-reference-docs@v0.3.1-0.20220223025230-af7c5e0048a3


Minor: Missing TODO/note to update this when a new release is tagged?

matthchr · 2022-03-04T21:19:28Z

scripts/gen-api-docs.sh

+    GROUPNAME=$(basename $(dirname $package))
+
+    #Filter the main CRD packages
+    if [[ $PACKAGE_VERSION =~ $PATTERN ]] && [[ "$PACKAGE_VERSION" != *"storage" ]] 


Maybe leave a comment explaining that here?

matthchr · 2022-03-04T21:20:34Z

v2/tools/generator/internal/astmodel/package_definition.go

 		err = emitGroupVersionFile(p, outputDir)
 		if err != nil {
 			return 0, err
 		}
+
+		// If package path does not include crossplane api dir and is not Storage package, then we generate doc file for the package


minor: Fix comment as crossplane is not longer explicitly checked here

matthchr · 2022-03-04T21:31:31Z

v2/tools/generator/internal/astmodel/package_definition.go

 	buf := &bytes.Buffer{}
-	err := groupVersionFileTemplate.Execute(buf, pkgDef)
+	defer buf.Reset()


Really? If you look at the Reset() implementation:

// Reset resets the buffer to be empty, // but it retains the underlying storage for use by future writes. // Reset is the same as Truncate(0). func (b *Buffer) Reset() { b.buf = b.buf[:0] b.off = 0 b.lastRead = opInvalid }

You can see there's no actual space savings here: https://go.dev/play/p/zXGZkI8C3Rv. Even if this did actually deallocate space, I don't think there's anything that makes a difference between waiting til the GC comes to clean up the slice whose space you deallocated, vs waiting for the GC to come clean up the buf itself. AFAIK both will take the same amount of time. @Porges might know something I don't there though.

matthchr · 2022-03-04T21:32:13Z

v2/tools/generator/internal/config/configuration.go

@@ -53,6 +53,8 @@ type Configuration struct {
 	// SamplesURL is the URL the samples are accessible at. Paths will be appended to the end of this to
 	// build full sample links. If this is not specified, no samples links are generated.
 	SamplesURL string `yaml:"samplesUrl"`
+	// EmitDocFiles is used as a signal to create doc.go files for packages


minor: Consider saying "the default is false" (even though it's sorta obvious because this is a non-ptr boolean).

Harshdeep Singh added 6 commits February 23, 2022 01:37

Add support for doc.go file creation in code generator; Add doc.go fi…

e691f32

…les for CRDs

Add doc generation task and script

bc38240

Add api docs; Add api docs templates

46f973f

Add doc.go for v2/api/resources

eb35c8a

Update docs/api/template/pkg.tpl to remove package links

e28c7d3

Rename CRD doc files

d576c08

super-harsh requested review from davefellows, theunrepentantgeek, Porges, matthchr and babbageclunk as code owners February 25, 2022 01:10

Harshdeep Singh and others added 10 commits February 27, 2022 22:12

Update check-changes.sh to ignore gen-api-docs.sh

7deb857

Update hugo/config.yaml to render crd api htlm text docs

127d837

Merge branch 'main' into feature/gen-api-docs

86955c7

Add empty lines in the beginning in authorization.v1alpha1api20200801…

6bb51d0

…preview.md

Merge branch 'feature/gen-api-docs' of https://github.com/Azure/azure…

9f1b553

…-service-operator into feature/gen-api-docs

Add extra lines in the beginning of the crd api docs file to fix 'pla…

d0f7b51

…in text html' hugo error

Revert hugo config.yaml

f42f3c5

Delete docs/api/template/placeholder.go

99f8f03

Add crossplane resource type check in doc.go file creation

1a52821

Add crossplane resource type check in doc.go file creation

d7a7595

matthchr reviewed Mar 2, 2022

View reviewed changes

Harshdeep Singh and others added 5 commits March 2, 2022 16:08

Update CRD API docs to have genruntime reference;

d06a1b8

Revert docs/hugo/config.toml

1a9a1dc

Add emitDocFiles signal to avoid creating docs for crossplane apis

6c26628

Merge branch 'main' into feature/gen-api-docs

effbe69

matthchr approved these changes Mar 4, 2022

View reviewed changes

Update comments; remove buf.Reset() from package_definition.go

e9fb5ef

super-harsh merged commit 79b8124 into main Mar 6, 2022

super-harsh deleted the feature/gen-api-docs branch March 6, 2022 22:58

matthchr added this to the codegen-beta-0 milestone Mar 7, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Autogenerate CRD docs #2116

Autogenerate CRD docs #2116

super-harsh commented Feb 25, 2022 •

edited

Loading

codecov-commenter commented Feb 27, 2022 •

edited

Loading

matthchr Mar 2, 2022

super-harsh Mar 2, 2022

matthchr Mar 4, 2022

matthchr Mar 2, 2022

super-harsh Mar 2, 2022 •

edited

Loading

matthchr Mar 4, 2022

matthchr left a comment

matthchr Mar 4, 2022

matthchr Mar 4, 2022

matthchr Mar 4, 2022

matthchr Mar 4, 2022

matthchr Mar 4, 2022

Autogenerate CRD docs #2116

Autogenerate CRD docs #2116

Conversation

super-harsh commented Feb 25, 2022 • edited Loading

codecov-commenter commented Feb 27, 2022 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

super-harsh Mar 2, 2022 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

matthchr left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

super-harsh commented Feb 25, 2022 •

edited

Loading

codecov-commenter commented Feb 27, 2022 •

edited

Loading

super-harsh Mar 2, 2022 •

edited

Loading