Can't inject on resources with the conduit.io/plane label.

[olix0r]

Running a command like:

:; target/cli/macos/conduit install -n inception -v $(bin/root-tag) |target/cli/macos/conduit inject -v $(bin/root-tag) - |kubectl apply -f -

I'd expect us to add the proxy to all conduit pods in the inception namespace. Unfortunately, we use a single label, conduit.io/plane to indicate whether the node is in the control plane.

It shouldn't be a requirement that a pod is only in one plane.

Furthermore, we should be able to run inject on pods that already have been injected (i.e. to update the proxy configuration).

I propose moving these labels to annotations so that they are not mutually exclusive.

Something like conduit.io/data-plane conduit.io/control-plane?

[siggy]

Proposed changes

Summary

conduit.io/plane: control => conduit.io/controller-component: web
conduit.io/controller: conduit => conduit.io/controller-ns: conduit
conduit.io/plane: data => (remove, redundant with conduit.io/controller-ns)

Before

conduit.io/plane: control: app is a member of the control plane
conduit.io/plane: data: app has an injected data plane
conduit.io/controller: conduit: app is connected to a controller deployed in the conduit namespace

conduit controller

annotations:
  conduit.io/created-by: conduit/cli git-7399df83
labels:
  app: controller
  conduit.io/plane: control

injected app

annotations:
  conduit.io/created-by: conduit/cli git-7399df83
  conduit.io/proxy-version: git-7399df83
labels:
  app: emoji-svc
  conduit.io/controller: conduit
  conduit.io/plane: data

After

conduit.io/controller-component: web: app is the web component of the control plane
conduit.io/controller-ns: conduit: app has an injected data plane, and is connected to a controller deployed in the conduit namespace

conduit controller

annotations:
  conduit.io/created-by: conduit/cli git-7399df83
labels:
  app: web
  conduit.io/controller-component: web

injected app

annotations:
  conduit.io/created-by: conduit/cli git-7399df83
  conduit.io/proxy-version: git-7399df83
labels:
  app: emoji-svc
  conduit.io/controller-ns: conduit

[olix0r]

I think conduit.io/mesh might be slightly clearer as conduit.io/controller-ns or something similar that explicitly indicates what the value is. Mesh is a bit of an underdefined term...

The member thing is a little unfortunate -- that we have a label with only one value. I wonder if we would benefit from using labels like conduit.io/controller-component: web conduit.io/controller-component: prometheus conduit.io/controller-component: controller, etc.

This enables label selections like conduit.io/controller-component to simply determine which pods are acting as a part of a controller, as well as things like conduit.io/controller-component != web` to enumerate non-web components, etc.

TIOLI.

generally seems fine to me.

[siggy]

Proposal updated with:

• conduit.io/controller-component
• conduit.io/controller-ns

[briansmith]

It shouldn't be a requirement that a pod is only in one plane.

It isn't clear to me what you are trying to accomplish with the "inception" example. Could you explain more what you're trying to do?

Furthermore, we should be able to run inject on pods that already have been injected (i.e. to update the proxy configuration).

I agree.

I propose moving these labels to annotations so that they are not mutually exclusive.

Something like conduit.io/data-plane conduit.io/control-plane?

We need these things to be labels because the Kubernetes API is good at selecting things based on labels and bad at selecting things based on any other criteria. For example, it is easy and efficient to find everything with the "conduit.io/control-plane" label. I don't know that it is easy and efficient to find everything with a similar annotation.

I think we do want to have the Conduit proxy injected into the control plane pods, however I don't think conduit inject should do that. Instead conduit install and/or other self-maintenance tasks in the control plane should manage the control plane's proxy configuration.

[briansmith]

annotations:

  conduit.io/created-by: conduit/cli git-7399df83
labels:
  app: web
  conduit.io/controller-component: web

injected app

annotations:
  conduit.io/created-by: conduit/cli git-7399df83
  conduit.io/proxy-version: git-7399df83
labels:
  app: emoji-svc
  conduit.io/controller-ns: conduit

I think this new scheme is fine. However, I would like to prevent conduit inject on control plane components unless/until we have a clearer plan for it. I think we should have the Conduit proxy injected into some or all of the control plane pods, but whether that happens should be controlled by Conduit and not be something decided by the end-user.

(To be crystal clear, the app labels aren't actually injected by Conduit, right?)

[siggy]

@briansmith Agree on all points.

It isn't clear to me what you are trying to accomplish with the "inception" example. Could you explain more what you're trying to do?

This is really just a proof of concept. Agree we can leverage the proxy between the control plane components, but it should not be left to the user to define that configuration.

We need these things to be labels because the Kubernetes API is good at selecting things based on labels and bad at selecting things based on any other criteria.

Yup, we're sticking with labels. My proposal above is what we're planning to implement.

(To be crystal clear, the app labels aren't actually injected by Conduit, right?)

Correct, I left the app label in there to help with context, it's not injected.

[briansmith]

OK, perfect.

[briansmith]

There's already an another open issue, #311, for injecting the proxy into the control plane pods and this issue already had a PR landed for it, so I'm closing this.