Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Design: Dynamic registration of admission controllers and initializers #611

Merged

Conversation

caesarxuchao
Copy link
Member

A sub-proposal of #132, proposal on Extensible Admission Control.

@smarterclayton @lavalamp @whitlockjc as initial reviewers.

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label May 10, 2017
@caesarxuchao caesarxuchao force-pushed the dynamic-initializer-admission branch from 4b6389a to b81ec0a Compare May 10, 2017 00:33

Instead of having the two controllers do consistent read of the entire
`AdmissionControlConfiguration` object, we let the registry store the
resourceVersion of the `AdmissionControlConfiguration` (perhaps in a configMap),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is no atomic update guarantee right? Apiserver can crash after updating the configmap but before updating AdmissionControlConfiguration.
We should be fine if we always update the configmap first.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, apiserver should always update the configmap first.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, apiserver would check for changes and rebuild the configmap on startup if necessary.

Copy link
Member

@lavalamp lavalamp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the substantive point here is to reconcile how we locate the webhooks with how the aggregator locates user apiservers.

type ExternalAdmissionHook struct {
// Operations is the list of operations this hook will be invoked on - Create, Update, or *
// for all operations. Defaults to '*'.
Operations []string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be an enum, not a string?

Operations []string
// Resources are the resources this hook should be invoked on. '*' is all resources.
Resources []string
// Subresources are the list of subresources this hook should be invoked on. '*' is all resources.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a subresource does not make sense without a resource? how does this work?

ClientConfig AdmissionHookClientConfig

// FailurePolicy defines how unrecognized errors from the admission endpoint are handled -
// allowed values are Ignore, Retry, Fail. Default value is Fail
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should retry be in at all? if yes, should it be in our first implementation?

type AdmissionHookClientConfig struct {
// Address of the external admission hook, could be a host string,
// a host:port pair, or a URL.
Address string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the aggregator uses a service instead of directly listing an address.

// Address of the external admission hook, could be a host string,
// a host:port pair, or a URL.
Address string
// ClientCertificate is the path to a client cert file for TLS.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This cannot include a client cert. Not as a file path, that doesn't make sense in a dynamic config object (won't work in envs where users don't have access to apiserver disk, which is half of the point of the dynamic config).

Instead, we have to make available a CA that the webhook trusts.

// CertificateAuthority is the path to a cert file for the certificate authority.
CertificateAuthority string
// CertificateAuthorityData contains PEM-encoded certificate authority certificates. Overrides CertificateAuthority
CertificateAuthorityData []byte
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Keep this. Take a look at the aggregation registration api types.

// If timeout is reached, the intializer is removed from the resource's
// initializer list by the apiserver.
// Default to XXX seconds.
Timeout *int64
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

a per-resource per-initializer timeout seems extreme. Better to have a global timeout?

If the `initializer admission controller` and the `generic webhook admission
controller` watch the `AdmissionControlConfiguration` and act upon deltas, their
cached version of the configuration might be arbitrarily delayed. This makes it
impossible to predicate what initializer/hooks will be applied to newly created
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/predicate/predict/


Instead of having the two controllers do consistent read of the entire
`AdmissionControlConfiguration` object, we let the registry store the
resourceVersion of the `AdmissionControlConfiguration` (perhaps in a configMap),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, apiserver would check for changes and rebuild the configmap on startup if necessary.

We considered a few ways to make the behavior of the `initializer admission
controller` and the `generic webhook admission controller` predictable.

(I prefer #2. #1 is inefficient, #3 requires complex schema and is not intuitive)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the clear specification of the options. IMO 2 is an optimization of 1, we should see if 1 is really problematic before doing the extra work. I would guess that an extra trip to etcd is going to be an insignificant amount of latency compared to the initializers/webhooks themselves. @smarterclayton agree?

@wojtek-t This is as good a place as any to ask about how this is going to impact our SLO, whether the 1 second return SLO will still make sense for create requests.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lavalamp - thanks for looping me in.

So I added a question about #2: https://github.com/kubernetes/community/pull/611/files#r116245850 which I think I don't understand.

The additional read from etcd for every create request sounds problematic, but we need to do some math to compute how many QPS it will generate.
I think that the only thing we are afraid about it latency of the call to etcd. We probably should care mostly about larger clusters. So let's say we are targeting 100pods/s throughput. So there will be O(100) create requests then.
In 5000-node clusters, we have 500 writes/s of just Node updates, so it's not that in such clusters we will have 10x more requests - it will be more like 10-20%. Still a lot though.

@caesarxuchao caesarxuchao force-pushed the dynamic-initializer-admission branch from e9479cb to 9d02a4a Compare May 10, 2017 23:29
Operations []OperationType
// Resources are the resources this hook should be invoked on. '*' is all resources.
Resources []string
// Subresources is list of subresources. If non-empty, this hook should be invoked on
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@smarterclayton i changed the comment. Does it match what you have in mind?

// for all operations. Defaults to '*'.
Operations []OperationType
// Resources are the resources this hook should be invoked on. '*' is all resources.
Resources []string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is API group specified?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated to include the group name.

@caesarxuchao
Copy link
Member Author

@lavalamp @smarterclayton ptal. Thanks.

type AdmissionHookClientConfig struct {
// Address of the external admission hook, could be a host string,
// a host:port pair, or a URL.
Address string
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@caesarxuchao caesarxuchao changed the title Proposal: Dynamic registration of admission controllers and initializers Design: Dynamic registration of admission controllers and initializers May 12, 2017
modifications.

```golang
type AdmissionControlConfiguration struct {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had assumed that we would want to allow components to register initializers independently. I.e. the component that provides "foo" initializer might be completely decoupled from initializer "bar", and therefore want to register in a decoupled way. That however assumes that two components can lazily define "before" and "after".

I think in general, most initializers don't care about before or after, and simply want a chance to fire. For those that do... what should we do?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought we would never trust initializers to register themselves. I thought it would be a human admin who wrote ACC. A single careless initializer could destroy the carefully ordered list of initializers, right?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just thinking that most initializers are going to be like 3rd party controllers. They're independently delivered. If we have one resource, the install steps for those controllers are going to be ugly patch statements. If we have multiple resources, we'll have to declare dependencies.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. In the future, will we want to enforce order among webhooks? If not, we can make the webhook part of ACC multiple instances.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think webhook "phases" or "before or after" are inevitable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a single ACC is much easier if both intilaizers and webhooks will have order.

If the initializer and webhook wish to do self registration, they can specify partial order with the latest strategic merge patch syntax.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would that look like?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

-1 for using PATCH for installation.

I think we probably eventually want the user to POST all the initializers/webhooks, and then a controller assembles the ordered list. However I think the thing Chao has here will be a lot easier to implement in the remaining time. Having this list here also lets us push off modeling the dependency graph into the future. So I propose to keep it as is for 1.7, even though it means installing these things will be a bit harder.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to https://github.com/kubernetes/community/pull/537/files, it would look like:

resourceInitializers:
  - resource
    $setElementOrder/initializers:
    - existing_initialier_a
    - existing_initialier_b
    - new_initializer
    - existing_initialier_c
    initilizers:
    - name: new_initiailizer
      failurePolicy: "ignore"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm ok with harder for now.

before applying the configuration to the incoming objects. This adds latency to
every CREATE request. Because the two admission controllers are in the same
process as the apiserver, the latency mainly consists of the consistent read
latency of the backend storage (etcd), and the proto unmarshalling.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fairly expensive today.

Thinking about this... watch from within the same etcd server is effectively free. We need to guarantee that the clock time for any given ACC is within some bound (because an admission controller can be arbitrarily delayed between when the data is read from etcd and when it applies the rules, so a hard boundary is probably too hard).

The scenario of greatest concern is when an apiservers watch client is delayed relative to the mutation operations being executed, perhaps due to a network hang. If we had a single etcd watch on a set of key ranges, our minimum bound is:

The most recent admission control configuration delivered on the same watch channel as any returned event on that watch channel must be no older than X seconds after the last update operation on this server was performed, otherwise a consistent read must be performed.

That can degrade to just performing a consistent read from each api server every X seconds and delaying / rejecting requests until the read returns.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Said another way, our etcd client simply has to ensure that the open watch channel from the server that also watches ACC has delivered ANY event within the time window we care about.

Copy link
Contributor

@smarterclayton smarterclayton May 12, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if the etcd client and server preserve the guarantee that would be necessary for the optimization there though. The client has to construct a watch, and the etcd server has to guarantee a "delivered before" behavior over the connection (event on ACC key has to be delivered before another key with higher index). gRPC substreams (one for each watch) might not support that if two watches are in different sub streams because the packets for the one stream can be delayed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or the api server write to the endpoints object has to track the member observed high etcd index and delay writes if other members of the cluster have higher observed indices (if the watch cache value is higher than the read value of the GET on endpoints)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I kind of like the upper bound and the degrading thing. Will make e2e tests slow though.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So from first principles:

  1. admin must be able to know when config is applied
  2. admin can always restart an apiserver and guarantee it sees the latest config (fallback)
  3. admin needs to be able to ensure config eventually applies within some bound

Those are good goals for an alpha api. To just do 3, could we:

  1. do a consistent read every 30s
  2. if the consistent read fails, block the admission plugin (fail) until it succeeds

Tests would have to get clever about verifying the config is working. If we're concerned, we can simply add a watch on the config objects and always use the latest, but the consistent read must succeed every 30s. That would allow tests to work most of the time.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is 2 an explicit goal? That sounds very easy to achieve.

Sure. I'll add this option to the doc, and let's see if we are on the same page.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just mean as - it's the thing that they have to do today, and it's worked so far. So it's ok to say "you must restart your masters to guarantee the latest config applied", which is probably reasonable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@smarterclayton - got it. Thanks a lot for explanation. That makes perfect sense to me.
I added a potential improvement for it in #611 (comment)

Instead of having the two controllers do consistent read of the entire
`AdmissionControlConfiguration` object, we let the registry store the
resourceVersion of the `AdmissionControlConfiguration` (perhaps in a configMap),
and let the two controllers always do consistent read of the resourceVersion and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I think I don't understand this option. Where is the "resourceVersion of the ACC" coming from? Do we still get it from etcd?
If not (just from local cache), I don't understand how it would work e.g. in HA setup.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The apiserver that handles the ACC update will persist the updated ACC's rv in a configmap. When updating the rv, the apiserver needs to ensure its increasing.

We propose two ways to make the behavior of the `initializer admission
controller` and the `generic webhook admission controller` predictable.

#### 1. Do consistent read of AdmissionControlConfiguration periodically
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wojtek-t how do you like this approach suggested by @smarterclayton ? It should be much lighter.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's start out with this option. Seems way easier. It is not super important that these things get applied instantly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@caesarxuchao - I'm completely fine with this option.
I would even go step further, and do:

  • let's do consistent read every 1s
  • if there wasn't any successful read over last 5s (or 10s) then block everythin

1qps will not be visible and this will give us some tolerance on unsuccessful reads (which may happen from time to time).

WDYT?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wojtek-t it sounds good. I'll incorporate it to the doc.

@caesarxuchao
Copy link
Member Author

@smarterclayton i incorporated your comments to the doc, ptal, thanks.

Copy link
Member

@lavalamp lavalamp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looking good, just a couple points

// Address of the external admission hook, could be a host string,
// a host:port pair, or a URL.
Address string
// Service is a reference to the service for this webhook. It must communicate
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say-- if multiple ports are exposed, it will use 443 if available and be an error condition otherwise. If only a single port is exposed it can just use that.

We propose two ways to make the behavior of the `initializer admission
controller` and the `generic webhook admission controller` predictable.

#### 1. Do consistent read of AdmissionControlConfiguration periodically
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's start out with this option. Seems way easier. It is not super important that these things get applied instantly.

to introduce the additional concept of "readiness".


## Considered bug REJECTED synchronization mechinism:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

s/bug/but/

the configuration to an incoming request. If the configmap has changed, then
they do a consistent read of the `AdmissionControlConfiguration`.

## What if an initializer controller/webhook is not ready after registered? (**optional for alpha implement**)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is unclear to me-- is the first step going to create things fail-open or fail-closed?

(I think we should do fail-open for 1.7 and add fail-closed behavior later when this feature grows up)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As long as it is explicitly specified that it is fail open

@caesarxuchao
Copy link
Member Author

@smarterclayton ptal. I'll start the implementation after your approval.

// apiserver removes initilizer from the initializers list of the resource
// if the timeout is reached; If "Fail" is set, apiserver returns timeout
// error if the timeout is reached.
FailurePolicy FailurePolicyType
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@smarterclayton do you agree with adding FailurePolicy to initializers? It's not in your original proposal #132.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use case here is "the system doesn't grind to a halt"?

Who in this scenario reaps these? Without this, initializers don't need a controller (or we can avoid it). With it, we must have one.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use case here is "the system doesn't grind to a halt"?

Yes.

Yes, we'll need a controller. I thought we could let apiserver do the reaping, but lavalamp pointed out apiserver could crash.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Or we can say initializers always fail closed for 1.7.

@caesarxuchao caesarxuchao force-pushed the dynamic-initializer-admission branch from e6f03ee to b3ada68 Compare May 15, 2017 19:45
The `initializer admission controller` and the `generic webhook admission
controller` do a consistent read of the configmap *everytime* before applying
the configuration to an incoming request. If the configmap has changed, then
they do a consistent read of the `AdmissionControlConfiguration`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Section for future work here?

@lavalamp
Copy link
Member

lavalamp commented May 15, 2017 via email

// If timeout is reached, an intializer is removed from the resource's
// initializer list by the apiserver.
// Default to XXX seconds.
Timeout *int64
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does a per-resource timeout make sense?

Does the timeout need to be customizable at all? What bad thing happens if we hard-code 30s?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will we ever have an initializer that takes longer than 30s?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can start with a global default of 30s. In Clayton's proposal i don't see any existing mutating admission controller that's going to take longer than 30s.

@whitlockjc
Copy link

I'll give the new version a peek over the weekend.

@caesarxuchao caesarxuchao force-pushed the dynamic-initializer-admission branch from dd9d024 to 950375f Compare May 20, 2017 00:02
@caesarxuchao
Copy link
Member Author

@whitlockjc @lavalamp @smarterclayton @deads2k ptal. Thanks!

// ExternalAdmissionHooks is a list of external admission webhooks and the
// affected resources and operations.
// +optional
ExternalAdmissionHooks []ExternalAdmissionHook `json:"externalAdmissionHooks,omitempty" patchStrategy:"merge" patchMergeKey:"name"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

webhooks are never ordered with respect to initializers, so why are these in the same object?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ack

// fails to takes action. Allowed values are Ignore, or Fail. If "Ignore" is
// set, initializer is removed from the initializers list of an object if
// the timeout is reached; If "Fail" is set, apiserver returns timeout error
// if the timeout is reached.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the default value?

// Name is the identifier of the initializer. It will be added to the
// object that needs to be initialized.
// Required
Name string `json:"name"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's start with forcing these to be fully qualified names. At least two dots: openshift.io should be illegal, but scc.openshift.io should be legal

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean enforcing the rule via validation?

type ExternalAdmissionHook struct {
// The name of the external admission webhook.
// Required.
Name string `json:"name"`
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same qualified name comment.

@caesarxuchao
Copy link
Member Author

Addressed @deads2k 's comments.

Copy link
Member

@lavalamp lavalamp left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

type Initializer struct {
// Name is the identifier of the initializer. It will be added to the
// object that needs to be initialized.
// Name should be fully qualified.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

give example?

// APIVersions is the API versions the resources belong to. '*' is all versions.
// If '*' is present, the length of the slice must be one.
// Required.
APIVersions []string `json:"apiVersions,omitempty"`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if I set groups to [apps, batch, core] and versions to [v1, v2alpha1, v1beta1], and not all versions are in all groups?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's recommended to separate them to different Rules, and make sure in each Rule all the tuple expansions make sense. The best practice is documented in the struct comment.

Also, even if the best practice is not followed, the Rule still works, the invalid tuple expansion is ignored.

@lavalamp lavalamp self-assigned this May 22, 2017
@lavalamp
Copy link
Member

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 22, 2017
@smarterclayton
Copy link
Contributor

smarterclayton commented May 23, 2017 via email

@smarterclayton smarterclayton merged commit f929dac into kubernetes:master May 23, 2017
// for all verbs.
// If '*' is present, the length of the slice must be one.
// Required.
Verbs []OperationType `json:"verbs,omitempty"`

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We are calling these Verbs but they are not, they are admission.Operations. I think we should be consistent with admission.Attributes and use Operations as the struct member name.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, why isn't Operations just a member of Rule instead of being wrapped in another type?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rbac called it verbs. I think being consistent with rbac is more important, because it's user-facing.

The rule doesn't contain verbs because it's used by initializers which only cares about CREATE.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think this is the case here. If someone were writing a Go-based AC, they would be operating with admission.Attributes and since our rules will filter/match based on admission.Attributes, I think that is the contract. I get that RBAC consistency is nice but I think this is one case where it's more confusing to do so.

/cc @deads2k @smarterclayton

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@whitlockjc you are right. I thought authorizer interface and admission interface would use the same term, but the are not, authorizer interface's method is called GetVerb, while the admission interface's method is called GetOperation. Is that intentional?

Rule `json:",inline"`
}

type OperationType string

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really don't see the purpose of mirroring admission.Operation here. I don't see it happening but what happens if an admission.Operation is modified or added or removed? Why not just use admission.Operation as the type?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let the api depends on the admission package sounds wrong. For example, if we decide to move the API to k8s.io/api, i don't want to make k8s.io/api depends on k8s.io/apiserver.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But since we're building against/for admission controllers, why is depending on admission wrong?

/cc @deads2k @smarterclayton

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It will create import cycle.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, that would depend on where we put these new types.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see you are already doing it in pkg/apis/admission: https://github.com/kubernetes/kubernetes/blob/master/pkg/apis/admission/types.go#L52

I think reversing the dependency is better, i.e. putting the definition in the API definition, because the API packages are usually the leaves in the dependency graph.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm still confused as to what goes into an API group vs. what ends up elsewhere. I could see some of these things in plugin/pkg/admission ending up in pkg/apis/admission myself. As for dependency stuff, I'd rather a stable object model than one that has multiple sources of truth. I realize that the chances of this are low right now but that doesn't mean impossible. Having admission.Operation being used by the internal admission stuff and having a shadow type to reflect it seems like one link that's easy to break. Just my two cents.

@whitlockjc
Copy link

I had a couple of consistency issues where our new types deviate from the types in admission. Since we are working with admission.Attributes objects, I think we should have consistent naming and types instead of new members/types that do not match their admission equivalent.

@smarterclayton
Copy link
Contributor

smarterclayton commented May 24, 2017 via email

smarterclayton pushed a commit to smarterclayton/kubernetes that referenced this pull request May 26, 2017
…ation-prototype

Automatic merge from submit-queue (batch tested with PRs 46383, 45645, 45923, 44884, 46294)

Dynamic registration prototype

Implementing the api proposed in kubernetes/community#611.
Wiring the code to serve the api via apiserver.
```release-note
Adding admissionregistration API group which enables dynamic registration of initializers and external admission webhooks. It is an alpha feature.
```
k8s-github-robot pushed a commit to kubernetes/kubernetes that referenced this pull request May 27, 2017
Automatic merge from submit-queue

Configuration manager for dynamic admission control registration

Implementing this [section](https://github.com/kubernetes/community/blob/master/contributors/design-proposals/dynamic-admission-control-configuration.md#synchronization-of-admission-control-configurations) of kubernetes/community#611

Adding a configuration manager that reads the ExternalAdmissionHookConfigurations and InitializerConfigurations periodically, and returns the merged configuration.

cc @smarterclayton @whitlockjc
MadhavJivrajani pushed a commit to MadhavJivrajani/community that referenced this pull request Nov 30, 2021
…er-admission

Design: Dynamic registration of admission controllers and initializers
danehans pushed a commit to danehans/community that referenced this pull request Jul 18, 2023
* add the release manager for Istio 1.11

* sort the release manager name according to alpha order
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants