Adopt aggregated apiserver #1048

aylei · 2019-10-24T17:25:56Z

Signed-off-by: Aylei rayingecho@gmail.com

What problem does this PR solve?

#1002

What is changed and how does it work?

Add aggregated apiserver skeleton and toolchain.

More todos are listed in the original issue #1002

Check List

Tests

Manual test

Create
Update
List
Watch

Code changes

Has Go code change

Does this PR introduce a user-facing change?:

New component TiDB aggregated apiserver.

Here's an example about adding new resources in AA: aylei#1

To make review easier, I split this PR to multiple commits:

Pin the versions of dependencies for Aggregated Apiserver, which actually can be ignored: 9103ca8
Add dataresource CRD, which introduce a new CRD called DataResource as the storage of our AA: 9103ca8
Generate code for dataresouce CRD, which can be ignored: 92fad18
Important: Implement a kube-apiserver based storage.Interface: 209a6e1
Add aggregated apiserver skeleton and toolchain: 0fa2977

PTAL @weekface @cofyc @tennix @onlymellb @Yisaer @DanielZhangQD

aylei · 2019-10-24T17:27:17Z

docs/adding-resource-in-aa.md

@@ -0,0 +1,90 @@
+# Adding New Resources in TiDB Aggregated Apiserver


A brief introduction of the usage

aylei · 2019-10-24T17:32:00Z

pkg/apiserver/cmd/start.go

+}
+
+// StartApiServer starts an apiserver.
+func StartApiServer(


AA entrypoint

aylei · 2019-10-24T17:33:09Z

pkg/apiserver/cmd/start.go

+
+// Config create an apiserver config
+func (o ServerOptions) Config() (*apiserver.Config, error) {
+	if err := o.RecommendedOptions.SecureServing.MaybeDefaultWithSelfSignedCerts(


The CA issue is left for later resolution

self-sign certs and skip tls verify should be enough for internal use

aylei · 2019-10-24T17:37:45Z

pkg/apiserver/storage/store.go

+)
+
+// store implements a ConfigMap backed storage.Interface
+type store struct {


Implementation of a kube-apiserver based storage.Interface, however, the implementation of watch is naive and will be improved in an separated PR.

add some unit tests to cover basic use cases?

aylei · 2019-10-24T17:39:20Z

/run-e2e-in-kind

aylei · 2019-10-24T17:48:13Z

e2e is not working for this issue now, because the apiserver lib introduce a dependency bitbucket.org/ww/goautoneg that have to be downloaded thru hg, which is not installed in the CI env.

$ go mod why bitbucket.org/ww/goautoneg
# bitbucket.org/ww/goautoneg
github.com/pingcap/tidb-operator/pkg/apiserver/cmd
k8s.io/apiserver/pkg/endpoints/filters
k8s.io/apiserver/pkg/endpoints/handlers/responsewriters
k8s.io/apiserver/pkg/endpoints/handlers/negotiation
bitbucket.org/ww/goautoneg

I will fix it soon, it does not block reviewing, though.

cofyc · 2019-10-25T03:44:42Z

/run-e2e-in-kind

cofyc · 2019-10-25T03:46:28Z

/run-e2e-in-kind

cofyc · 2019-10-25T03:48:14Z

e2e is not working for this issue now, because the apiserver lib introduce a dependency bitbucket.org/ww/goautoneg that have to be downloaded thru hg, which is not installed in the CI env.

I've added a command to install hg in CI script.

aylei · 2019-10-25T03:49:33Z

@cofyc Thanks!

cofyc · 2019-10-25T09:09:13Z

tools/tools.go

@@ -32,4 +32,5 @@ import (
 	_ "k8s.io/code-generator/cmd/openapi-gen"
 	_ "k8s.io/code-generator/cmd/register-gen"
 	_ "k8s.io/code-generator/cmd/set-gen"
+	_ "github.com/kubernetes-incubator/apiserver-builder-alpha/cmd/apiregister-gen"


why not use sigs.k8s.io/apiserver-builder-alpha/cmd/apiregister-gen?

this project is moved to https://github.com/kubernetes-sigs/apiserver-builder-alpha/ now.

The pinned version v1.12 was still using the kubernetes-incubator import path, we can use sigs.k8s.io.... after we bump the kubernetes dependencies to v1.16

cofyc · 2019-10-28T06:42:05Z

because I renamed pkg/apis/pingcap.com to pkg/apis/pingcap, some conflicts must be resolved. regenerating code and updating imports should be enough

aylei · 2019-10-28T06:49:17Z

@cofyc Thanks for noting!

Signed-off-by: Aylei <rayingecho@gmail.com>

pkg/apiserver/storage/store.go

pkg/apiserver/cmd/start.go

pkg/apis/pingcap.com/v1alpha1/dataresource_types.go

tennix

LGTM

Signed-off-by: Aylei <rayingecho@gmail.com>

aylei · 2019-10-29T12:39:10Z

/run-e2e-tests

aylei · 2019-10-29T12:42:19Z

@cofyc @tennix @onlymellb @DanielZhangQD I've addressed the review comments and add the basic unit tests for store.go(626f3f0). Please take a look again and approve if there's no problem, thanks!

/run-e2e-in-kind

Signed-off-by: Aylei <rayingecho@gmail.com>

aylei · 2019-10-29T13:48:11Z

/run-e2e-in-kind

Signed-off-by: Aylei <rayingecho@gmail.com>

aylei · 2019-10-29T15:17:09Z

/run-e2e-in-kind

aylei · 2019-10-30T01:43:38Z

/run-e2e-in-kind

cofyc · 2019-10-30T02:33:12Z

/run-e2e-in-kind

cofyc · 2019-10-30T02:37:03Z

/run-e2e-in-kind

aylei · 2019-10-30T06:49:39Z

/run-e2e-in-kind

tennix

LGTM

DanielZhangQD

LGTM

cofyc

LGTM

aylei requested review from weekface, cofyc, tennix and onlymellb October 24, 2019 17:25

aylei commented Oct 24, 2019

View reviewed changes

pkg/apiserver/cmd/start.go

}

// StartApiServer starts an apiserver.

func StartApiServer(

Copy link

Contributor Author

aylei Oct 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AA entrypoint

aylei commented Oct 24, 2019

View reviewed changes

aylei force-pushed the issue-1002 branch from 4509425 to 65cfa68 Compare October 24, 2019 17:34

aylei commented Oct 24, 2019

View reviewed changes

aylei mentioned this pull request Oct 24, 2019

Adopt aggregated apiserver #1002

Closed

10 tasks

cofyc reviewed Oct 25, 2019

View reviewed changes

tennix added the area/api label Oct 28, 2019

This was referenced Oct 28, 2019

Basic e2e tests for aggregated apiserver #1058

Closed

Install aggregated apiserver in the tidb-operator helm chart #1060

Closed

aylei requested review from DanielZhangQD and Yisaer October 28, 2019 07:32

aylei added 6 commits October 28, 2019 17:09

Pin dependencies for aggregated apiserver

9103ca8

Signed-off-by: Aylei <rayingecho@gmail.com>

Add dataresource CRD

597c6c2

Signed-off-by: Aylei <rayingecho@gmail.com>

Generate code for dataresource CRD

92fad18

Signed-off-by: Aylei <rayingecho@gmail.com>

Implement a kube-apiserver backed storage.Interface

209a6e1

Signed-off-by: Aylei <rayingecho@gmail.com>

Add dataresouce CRD yaml

f0cfe98

Signed-off-by: Aylei <rayingecho@gmail.com>

Add aggregated apiserver skeleton and toolchain

0fa2977

Signed-off-by: Aylei <rayingecho@gmail.com>

tennix reviewed Oct 29, 2019

View reviewed changes

pkg/apiserver/storage/store.go Show resolved Hide resolved

onlymellb reviewed Oct 29, 2019

View reviewed changes

pkg/apiserver/cmd/start.go Outdated Show resolved Hide resolved

onlymellb reviewed Oct 29, 2019

View reviewed changes

pkg/apiserver/cmd/start.go Outdated Show resolved Hide resolved

DanielZhangQD reviewed Oct 29, 2019

View reviewed changes

pkg/apis/pingcap.com/v1alpha1/dataresource_types.go Outdated Show resolved Hide resolved

tennix previously approved these changes Oct 29, 2019

View reviewed changes

tennix added the status/LGTM1 label Oct 29, 2019

aylei dismissed tennix’s stale review via f6e67da October 29, 2019 12:25

aylei added 2 commits October 29, 2019 20:25

Unittest for kube-apiserver backed store

626f3f0

Signed-off-by: Aylei <rayingecho@gmail.com>

Merge upstream/master

696355d

aylei force-pushed the issue-1002 branch from f6e67da to 626f3f0 Compare October 29, 2019 12:30

Address review comments

406278f

Signed-off-by: Aylei <rayingecho@gmail.com>

aylei mentioned this pull request Oct 29, 2019

upgrade k8s code to 1.16 #1020

Closed

Fix ci

6ba07ce

Signed-off-by: Aylei <rayingecho@gmail.com>

Abandon informer based read for now in favor of linearizability

7e55d6b

Signed-off-by: Aylei <rayingecho@gmail.com>

Merge branch 'master' into issue-1002

da8dbab

tennix approved these changes Oct 30, 2019

View reviewed changes

DanielZhangQD reviewed Oct 30, 2019

View reviewed changes

aylei requested review from cofyc, onlymellb and DanielZhangQD October 30, 2019 08:04

cofyc approved these changes Oct 30, 2019

View reviewed changes

aylei merged commit fa36736 into pingcap:master Oct 30, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adopt aggregated apiserver #1048

Adopt aggregated apiserver #1048

aylei commented Oct 24, 2019 •

edited

Loading

aylei Oct 24, 2019

aylei Oct 24, 2019

aylei Oct 24, 2019

cofyc Oct 29, 2019

aylei Oct 24, 2019

cofyc Oct 29, 2019

aylei commented Oct 24, 2019

aylei commented Oct 24, 2019 •

edited

Loading

cofyc commented Oct 25, 2019

cofyc commented Oct 25, 2019

cofyc commented Oct 25, 2019 •

edited

Loading

aylei commented Oct 25, 2019

cofyc Oct 25, 2019 •

edited

Loading

aylei Oct 28, 2019

cofyc commented Oct 28, 2019

aylei commented Oct 28, 2019

tennix left a comment

aylei commented Oct 29, 2019

aylei commented Oct 29, 2019

aylei commented Oct 29, 2019

aylei commented Oct 29, 2019

aylei commented Oct 30, 2019

cofyc commented Oct 30, 2019

cofyc commented Oct 30, 2019

aylei commented Oct 30, 2019

tennix left a comment

DanielZhangQD left a comment

cofyc left a comment

		@@ -0,0 +1,90 @@
		# Adding New Resources in TiDB Aggregated Apiserver

Adopt aggregated apiserver #1048

Adopt aggregated apiserver #1048

Conversation

aylei commented Oct 24, 2019 • edited Loading

What problem does this PR solve?

What is changed and how does it work?

Check List

Does this PR introduce a user-facing change?:

aylei Oct 24, 2019

Choose a reason for hiding this comment

aylei Oct 24, 2019

Choose a reason for hiding this comment

aylei Oct 24, 2019

Choose a reason for hiding this comment

cofyc Oct 29, 2019

Choose a reason for hiding this comment

aylei Oct 24, 2019

Choose a reason for hiding this comment

cofyc Oct 29, 2019

Choose a reason for hiding this comment

aylei commented Oct 24, 2019

aylei commented Oct 24, 2019 • edited Loading

cofyc commented Oct 25, 2019

cofyc commented Oct 25, 2019

cofyc commented Oct 25, 2019 • edited Loading

aylei commented Oct 25, 2019

cofyc Oct 25, 2019 • edited Loading

Choose a reason for hiding this comment

aylei Oct 28, 2019

Choose a reason for hiding this comment

cofyc commented Oct 28, 2019

aylei commented Oct 28, 2019

tennix left a comment

Choose a reason for hiding this comment

aylei commented Oct 29, 2019

aylei commented Oct 29, 2019

aylei commented Oct 29, 2019

aylei commented Oct 29, 2019

aylei commented Oct 30, 2019

cofyc commented Oct 30, 2019

cofyc commented Oct 30, 2019

aylei commented Oct 30, 2019

tennix left a comment

Choose a reason for hiding this comment

DanielZhangQD left a comment

Choose a reason for hiding this comment

cofyc left a comment

Choose a reason for hiding this comment

aylei commented Oct 24, 2019 •

edited

Loading

aylei commented Oct 24, 2019 •

edited

Loading

cofyc commented Oct 25, 2019 •

edited

Loading

cofyc Oct 25, 2019 •

edited

Loading