Add Agones Kubernetes API docs generator #645

aLekSer · 2019-03-12T15:17:13Z

Add gen-crd-api-reference-docs utility into build-image, generate Agones CRD reference html document, map this doc to existent docs structure.
Covers #409.

aLekSer · 2019-03-12T15:21:37Z

This is an initial commit. Some changes to the templates could occur in the future. Current look and feel:

agones-bot · 2019-03-12T15:30:26Z

Build Failed 😱

Build Id: 0d57e618-eea9-4b9a-a7d2-955b0ccb8f15

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

aLekSer · 2019-03-12T15:57:39Z

Issue with resulting file position in Cloud Build:

Output to a file /go/src/agones.dev/agones/site/content/en/docs/Reference/agones_crd_api_reference.html
diff /workspace/site/content/en/docs/Reference/agones_crd_api_reference.html /tmp/generated.html

Locally I have for test-gen-api-docs: expected_docs := $(site_path)/$(REL_PATH) :

diff /Users/alexander.apalikov/go/src/agones.dev/agones/site/content/en/docs/Reference/agones_crd_api_reference.html /tmp/generated.html

agones-bot · 2019-03-12T16:36:31Z

Build Failed 😱

Build Id: 7d2b8038-01f1-489c-9e06-38b1c8f6bb5b

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2019-03-12T16:45:51Z

Build Failed 😱

Build Id: ec4f0130-15f3-407e-8536-f52a56a023ac

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

aLekSer · 2019-03-12T20:53:37Z

Found an issue, I have added the special kubebuilder declaration in /pkg/apis/stable/v1alpha1/gameserver.go, which is redundant, like that:

// +kubebuilder:resource:path=gameservers

agones-bot · 2019-03-12T21:00:07Z

Build Succeeded 👏

Build Id: 96fa6cbf-bb93-4d46-8ea3-4b4bd8bbb781

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.9.0-b226b37
image: gcr.io/agones-images/agones-sdk:0.9.0-b226b37
image: gcr.io/agones-images/agones-ping:0.9.0-b226b37
Linux C++ SDK (build: dev): agonessdk-0.9.0-b226b37-dev-linux-arch_64.tar.gz
Linux C++ SDK (build: runtime): agonessdk-0.9.0-b226b37-runtime-linux-arch_64.tar.gz
Linux C++ SDK (source): agonessdk-0.9.0-b226b37-src.zip
SDK Server: agonessdk-server-0.9.0-b226b37.zip

A preview of the website (the last 30 builds are retained):

https://b226b37-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/645/head:pr_645 && git checkout pr_645
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.9.0-b226b37

markmandel

We should probably reference the new page on https://agones.dev/site/docs/guides/access-api/

Otherwise, this looks amazing! Had some questions below about how to manage the bleeding edge.

markmandel · 2019-03-13T00:48:12Z

build/includes/website.mk

@@ -24,6 +24,23 @@
 # Website targets
 #

+REL_PATH := content/en/docs/Reference/agones_crd_api_reference.html
+API_DOC_PATH := "$(mount_path)/site/$(REL_PATH)"
+GEN_API_DOCS ?= docker run -e FILE=$(API_DOC_PATH) --rm -i $(common_mounts) $(build_tag) bash -c "/go/src/agones.dev/agones/site/gen-api-docs.sh"


Curious as to the motivation to put all of this in variables? Seems like it wouldn't change?

I have added those options to be able to split test html result and normal one. GEN_API_DOCS used twice, that's why it was extracted.

markmandel · 2019-03-13T00:53:31Z

build/includes/website.mk

@@ -59,6 +76,7 @@ site-deploy-preview: site-static-preview

 site-test:
 	# generate actual html and run test against - provides a more accurate tests
+	$(MAKE) test-gen-api-docs


I don't have a good answer for this - but because the website works off master - if we change our CRD data structures, that will change the website as well, and be out of sync until next release.

Only idea I've got - that in the advent of that happening - rename the old page and put an expiryDate of the next release in the toml.... but how would we get a publishDate in the new one, and still pass tests?

I'm just thinking out loud here, but any ideas?

Tried to use custom short code in front-matter and it seems that variables are not supported there.
So currently I am thinking of combining one full generated html with expiryVersion shortcode. And new one with publishVersion shortcode in the same file.
And during test just compare the second half with publishVersion. And during release move bottom half to the top and generate new one at the bottom.

That sounds fancy - but makes sense to me! I assume you can make that work? 😄

I have updated gen-api-docs.sh script. It now handles base_version environment variable.
What it does is get all lines before publishVersion line. Then extract what in the middle of this feature shortcode. Compares with what was generated if diff then do version compare.
Then set expiryVersion of old part with new base_version fresh generated doc now have new publishVersion.

agones-bot · 2019-03-14T14:29:12Z

Build Failed 😱

Build Id: 75d62778-eb37-40d8-a9da-f1efd4fcafc9

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

aLekSer · 2019-03-14T14:29:15Z

Reworked generation script it now handles the version in publishVersion shortcode.
Old code becomes expiredVersion with new base_version. Test now compares only what is in the middle of publishVersion with the output of go tool gen-crd-api-reference-docs.
Two calls to awk and one sed do the job.
We don't need to care about switching the versions on release process - it is automated for this file.

agones-bot · 2019-03-14T14:40:55Z

Build Failed 😱

Build Id: 507ee387-19d4-4874-a52d-d1751d1e54cb

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2019-03-14T14:47:49Z

Build Failed 😱

Build Id: dd261584-206a-4d53-a568-3d419ea693c3

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2019-03-14T15:02:39Z

Build Succeeded 👏

Build Id: ff3a87dc-93b5-465d-8773-c8f900f38e91

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.9.0-2996acc
image: gcr.io/agones-images/agones-sdk:0.9.0-2996acc
image: gcr.io/agones-images/agones-ping:0.9.0-2996acc
Linux C++ SDK (build: dev): agonessdk-0.9.0-2996acc-dev-linux-arch_64.tar.gz
Linux C++ SDK (build: runtime): agonessdk-0.9.0-2996acc-runtime-linux-arch_64.tar.gz
Linux C++ SDK (source): agonessdk-0.9.0-2996acc-src.zip
SDK Server: agonessdk-server-0.9.0-2996acc.zip

A preview of the website (the last 30 builds are retained):

https://2996acc-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/645/head:pr_645 && git checkout pr_645
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.9.0-2996acc

agones-bot · 2019-03-16T07:20:40Z

Build Failed 😱

Build Id: ca835708-a565-4674-a317-627f3f0af47c

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2019-03-16T10:57:17Z

Build Succeeded 👏

Build Id: 615bafc8-a477-4be4-9411-534826f41cfe

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.9.0-69a97ad
image: gcr.io/agones-images/agones-sdk:0.9.0-69a97ad
image: gcr.io/agones-images/agones-ping:0.9.0-69a97ad
Linux C++ SDK (build: dev): agonessdk-0.9.0-69a97ad-dev-linux-arch_64.tar.gz
Linux C++ SDK (build: runtime): agonessdk-0.9.0-69a97ad-runtime-linux-arch_64.tar.gz
Linux C++ SDK (source): agonessdk-0.9.0-69a97ad-src.zip
SDK Server: agonessdk-server-0.9.0-69a97ad.zip

A preview of the website (the last 30 builds are retained):

https://69a97ad-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/645/head:pr_645 && git checkout pr_645
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.9.0-69a97ad

aLekSer · 2019-03-16T16:31:43Z

I think I need to add md instructions on how to use new make target gen-api-docs. It was created to work in a similar way to make gen-install when we call this on each crd update and just git add the resulted .../Reference/agones_crd_api_reference.html. So basically two subsequent calls of $(MAKE) test-gen-api-docs would result in pass the test, but on CI if someone forgot to commit this html than test would fail.

agones-bot · 2019-03-18T16:21:48Z

Build Succeeded 👏

Build Id: 50f9b339-45ab-4cf6-9ea2-0c84f8b0f855

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.9.0-363d979
image: gcr.io/agones-images/agones-sdk:0.9.0-363d979
image: gcr.io/agones-images/agones-ping:0.9.0-363d979
Linux C++ SDK (build: dev): agonessdk-0.9.0-363d979-dev-linux-arch_64.tar.gz
Linux C++ SDK (build: runtime): agonessdk-0.9.0-363d979-runtime-linux-arch_64.tar.gz
Linux C++ SDK (source): agonessdk-0.9.0-363d979-src.zip
SDK Server: agonessdk-server-0.9.0-363d979.zip

A preview of the website (the last 30 builds are retained):

https://363d979-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/645/head:pr_645 && git checkout pr_645
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.9.0-363d979

agones-bot · 2019-03-18T16:28:41Z

Build Succeeded 👏

Build Id: f83d51da-1af9-497d-8958-b857fd814143

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.9.0-87798f8
image: gcr.io/agones-images/agones-sdk:0.9.0-87798f8
image: gcr.io/agones-images/agones-ping:0.9.0-87798f8
Linux C++ SDK (build: dev): agonessdk-0.9.0-87798f8-dev-linux-arch_64.tar.gz
Linux C++ SDK (build: runtime): agonessdk-0.9.0-87798f8-runtime-linux-arch_64.tar.gz
Linux C++ SDK (source): agonessdk-0.9.0-87798f8-src.zip
SDK Server: agonessdk-server-0.9.0-87798f8.zip

A preview of the website (the last 30 builds are retained):

https://87798f8-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/645/head:pr_645 && git checkout pr_645
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.9.0-87798f8

aLekSer · 2019-03-19T09:44:03Z

Fixed one issue, there was a commit-id in a default template. So I removed it, so now different git commit would generate the same html.
https://github.com/GoogleCloudPlatform/agones/pull/645/files#diff-b75745dde8c5e52ecb85780a42ac3669

agones-bot · 2019-03-19T09:56:37Z

Build Succeeded 👏

Build Id: dca396f0-df09-43bd-b952-a267c69edc80

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.9.0-1d683f7
image: gcr.io/agones-images/agones-sdk:0.9.0-1d683f7
image: gcr.io/agones-images/agones-ping:0.9.0-1d683f7
Linux C++ SDK (build: dev): agonessdk-0.9.0-1d683f7-dev-linux-arch_64.tar.gz
Linux C++ SDK (build: runtime): agonessdk-0.9.0-1d683f7-runtime-linux-arch_64.tar.gz
Linux C++ SDK (source): agonessdk-0.9.0-1d683f7-src.zip
SDK Server: agonessdk-server-0.9.0-1d683f7.zip

A preview of the website (the last 30 builds are retained):

https://1d683f7-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/645/head:pr_645 && git checkout pr_645
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.9.0-1d683f7

markmandel · 2019-03-16T16:22:04Z

build/includes/website.mk

@@ -24,6 +24,22 @@
 # Website targets
 #

+REL_PATH := content/en/docs/Reference/agones_crd_api_reference.html
+GEN_API_DOCS ?= docker run -e FILE="$(mount_path)/site/$(REL_PATH)" -e VERSION=${base_version} --rm -i $(common_mounts) $(build_tag) bash -c "/go/src/agones.dev/agones/site/gen-api-docs.sh"


Super minor nit on this - should this be := rather than ?= , since I don't think anyone should be overwriting the GEN_API_DOCS command?

markmandel · 2019-03-16T16:24:17Z

site/content/en/docs/Guides/access-api.md

-which can be accessed and manipulated through the Kubernetes API.
+which can be accessed and manipulated through the Kubernetes API. 
+
+{{% feature publishVersion="0.9.0" %}}


I don't think the CRDs have changed - could we publish this all now? (0.8.0?)

Updated publishVersion, test still pass.

markmandel · 2019-03-25T00:50:30Z

Apologies - looks like I never submitted those comments :/ they've been sitting there forever. But otherwise, this is good to go.

aLekSer · 2019-03-25T09:49:21Z

Merged master, fixed publishVersion and Makefile parameter.

agones-bot · 2019-03-25T10:00:26Z

Build Failed 😱

Build Id: 22e8da75-dd1b-49d1-b804-ec69259ed6d7

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2019-03-25T10:19:12Z

Build Succeeded 👏

Build Id: fc9feedc-adba-4e18-b303-ce1eaff06f66

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.9.0-8e3b89f
image: gcr.io/agones-images/agones-sdk:0.9.0-8e3b89f
image: gcr.io/agones-images/agones-ping:0.9.0-8e3b89f
Linux C++ SDK (build: dev): agonessdk-0.9.0-8e3b89f-dev-linux-arch_64.tar.gz
Linux C++ SDK (build: runtime): agonessdk-0.9.0-8e3b89f-runtime-linux-arch_64.tar.gz
Linux C++ SDK (source): agonessdk-0.9.0-8e3b89f-src.zip
SDK Server: agonessdk-server-0.9.0-8e3b89f.zip

A preview of the website (the last 30 builds are retained):

https://8e3b89f-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/645/head:pr_645 && git checkout pr_645
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.9.0-8e3b89f

Add gen-crd-api-reference-docs utility into build-image, generate Agones CRD reference html document, map this doc to existent docs structure.

agones-bot · 2019-03-25T16:32:23Z

Build Succeeded 👏

Build Id: f9a616c7-3582-465a-832e-fb93fa661b61

The following development artifacts have been built, and will exist for the next 30 days:

image: gcr.io/agones-images/agones-controller:0.9.0-4a1b4a9
image: gcr.io/agones-images/agones-sdk:0.9.0-4a1b4a9
image: gcr.io/agones-images/agones-ping:0.9.0-4a1b4a9
Linux C++ SDK (build: dev): agonessdk-0.9.0-4a1b4a9-dev-linux-arch_64.tar.gz
Linux C++ SDK (build: runtime): agonessdk-0.9.0-4a1b4a9-runtime-linux-arch_64.tar.gz
Linux C++ SDK (source): agonessdk-0.9.0-4a1b4a9-src.zip
SDK Server: agonessdk-server-0.9.0-4a1b4a9.zip

A preview of the website (the last 30 builds are retained):

https://4a1b4a9-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/GoogleCloudPlatform/agones.git pull/645/head:pr_645 && git checkout pr_645
helm install install/helm/agones --namespace agones-system --name agones --set agones.image.tag=0.9.0-4a1b4a9

markmandel

Awesome! 👍

ahmetb · 2019-04-03T06:19:42Z

build/build-image/Dockerfile

@@ -123,6 +123,11 @@ RUN mkdir /tmp/hugo && \
 RUN curl -sL https://deb.nodesource.com/setup_11.x | bash - && \
    apt-get install -y nodejs

+# install API reference docs generator
+RUN mkdir -p /go/src/github.com/ahmetb && \
+    cd /go/src/github.com/ahmetb && git clone https://github.com/ahmetb/gen-crd-api-reference-docs && \


I think you should probably pin to a commit here.
Given there's a copy of the template in this repo, I don't want to accidentally break the downstream.

Thanks @ahmetb for the review. I was thinking about that, will provide an update in a separate PR.

aLekSer force-pushed the api-docs-generation branch from 562a2bb to bd24ae0 Compare March 12, 2019 16:30

aLekSer force-pushed the api-docs-generation branch from bd24ae0 to 790ac8d Compare March 12, 2019 16:39

aLekSer force-pushed the api-docs-generation branch from 790ac8d to b226b37 Compare March 12, 2019 20:48

markmandel reviewed Mar 13, 2019

View reviewed changes

aLekSer force-pushed the api-docs-generation branch from b226b37 to 16607a2 Compare March 14, 2019 14:22

aLekSer force-pushed the api-docs-generation branch 2 times, most recently from 4fa7575 to 9e6b85a Compare March 14, 2019 14:38

aLekSer force-pushed the api-docs-generation branch from 9e6b85a to 2996acc Compare March 14, 2019 14:51

aLekSer force-pushed the api-docs-generation branch from 2996acc to f711ddb Compare March 16, 2019 07:14

aLekSer force-pushed the api-docs-generation branch from f711ddb to 69a97ad Compare March 16, 2019 10:46

aLekSer force-pushed the api-docs-generation branch 2 times, most recently from 363d979 to 87798f8 Compare March 18, 2019 16:17

aLekSer force-pushed the api-docs-generation branch from 87798f8 to 1d683f7 Compare March 19, 2019 09:38

markmandel reviewed Mar 25, 2019

View reviewed changes

aLekSer force-pushed the api-docs-generation branch from 1d683f7 to 2b233fb Compare March 25, 2019 09:47

aLekSer force-pushed the api-docs-generation branch from 2b233fb to 8e3b89f Compare March 25, 2019 10:05

Add extended API docs generator

4a1b4a9

Add gen-crd-api-reference-docs utility into build-image, generate Agones CRD reference html document, map this doc to existent docs structure.

aLekSer force-pushed the api-docs-generation branch from 8e3b89f to 4a1b4a9 Compare March 25, 2019 16:06

markmandel approved these changes Mar 25, 2019

View reviewed changes

markmandel merged commit 7368392 into googleforgames:master Mar 25, 2019

markmandel added area/site The website itself - including redirects, styling, search, etc kind/documentation Documentation for Agones kind/feature New features for Agones labels Mar 26, 2019

markmandel added this to the 0.9.0 milestone Mar 26, 2019

ahmetb reviewed Apr 3, 2019

View reviewed changes

Add Agones Kubernetes API docs generator #645

Add Agones Kubernetes API docs generator #645

Conversation

aLekSer commented Mar 12, 2019

aLekSer commented Mar 12, 2019 • edited Loading

agones-bot commented Mar 12, 2019

aLekSer commented Mar 12, 2019

agones-bot commented Mar 12, 2019

agones-bot commented Mar 12, 2019

aLekSer commented Mar 12, 2019

agones-bot commented Mar 12, 2019

markmandel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

agones-bot commented Mar 14, 2019

aLekSer commented Mar 14, 2019

agones-bot commented Mar 14, 2019

agones-bot commented Mar 14, 2019

agones-bot commented Mar 14, 2019

agones-bot commented Mar 16, 2019

agones-bot commented Mar 16, 2019

aLekSer commented Mar 16, 2019

agones-bot commented Mar 18, 2019

agones-bot commented Mar 18, 2019

aLekSer commented Mar 19, 2019

agones-bot commented Mar 19, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aLekSer Mar 25, 2019 • edited Loading

Choose a reason for hiding this comment

markmandel commented Mar 25, 2019

aLekSer commented Mar 25, 2019

agones-bot commented Mar 25, 2019

agones-bot commented Mar 25, 2019

agones-bot commented Mar 25, 2019

markmandel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

aLekSer commented Mar 12, 2019 •

edited

Loading

aLekSer Mar 25, 2019 •

edited

Loading