Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[ExternalNode] Create Secret for vm-agent in RBAC #4560

Merged
merged 1 commit into from
Feb 2, 2023
Merged

[ExternalNode] Create Secret for vm-agent in RBAC #4560

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions build/yamls/externalnode/vm-agent-rbac.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,15 @@ metadata:
name: vm-agent
namespace: vm-ns # Change the Namespace to where vm-agent is expected to run.
---
apiVersion: v1
kind: Secret
metadata:
name: vm-agent-service-account-token
namespace: vm-ns # Change the Namespace to where vm-agent is expected to run.
annotations:
kubernetes.io/service-account.name: vm-agent
type: kubernetes.io/service-account-token
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
Expand Down
4 changes: 2 additions & 2 deletions ci/jenkins/test-vm.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,8 +193,9 @@ function create_kubeconfig_files {
echo "Creating files ${ANTREA_AGENT_KUBECONFIG} and ${ANTREA_AGENT_ANTREA_KUBECONFIG}"
# Kubeconfig to access K8S API

SECRET_NAME="${SERVICE_ACCOUNT}-service-account-token"
APISERVER=$(kubectl config view -o jsonpath="{.clusters[?(@.name==\"$CLUSTER_NAME\")].cluster.server}")
TOKEN=$(kubectl -n $TEST_NAMESPACE get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='$SERVICE_ACCOUNT')].data.token}"|base64 --decode)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any particular reason to change it to use Secret Name instead of ServiceAccount Name?

Copy link
Contributor Author

@wenyingd wenyingd Jan 16, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to be compatible with the original K8s versions. After we changed the RBAC file for VM Agent, a new Secret with name "vm-agent-service-account-token" is always created. It should work fine with newer versions in K8s cluster as there is only one Secret (what is created by us) is found with the annotation "kubernetes.io/service-account.name: vm-agent". For previous versions, a default Secret is created along with the SA, there would be two Secrets found with the annotation, then the final token string is incorrect as a concat of two tokens. As a result, I change the script to leverage the Secret which must exist in both previous version and later version.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good. Thanks for the clarification.

TOKEN=$(kubectl -n $TEST_NAMESPACE get secrets ${SECRET_NAME} -o json | jq -r .data.token | base64 --decode)
kubectl config --kubeconfig=${WORKDIR}/${ANTREA_AGENT_KUBECONFIG} set-cluster kubernetes --server=$APISERVER --insecure-skip-tls-verify=true
kubectl config --kubeconfig=${WORKDIR}/${ANTREA_AGENT_KUBECONFIG} set-credentials antrea-agent --token=$TOKEN
kubectl config --kubeconfig=${WORKDIR}/${ANTREA_AGENT_KUBECONFIG} set-context antrea-agent@kubernetes --cluster=kubernetes --user=antrea-agent
Expand All @@ -203,7 +204,6 @@ function create_kubeconfig_files {
# Kubeconfig to access AntreaController
ANTREA_API_SERVER_IP=$(kubectl get nodes -o wide --no-headers=true | awk -v role="$CONTROL_PLANE_NODE_ROLE" '$3 != role {print $6}')
ANTREA_API_SERVER="https://${ANTREA_API_SERVER_IP}:32767"
TOKEN=$(kubectl -n $TEST_NAMESPACE get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='$SERVICE_ACCOUNT')].data.token}"|base64 --decode)
kubectl config --kubeconfig=${WORKDIR}/${ANTREA_AGENT_ANTREA_KUBECONFIG} set-cluster antrea --server=$ANTREA_API_SERVER --insecure-skip-tls-verify=true
kubectl config --kubeconfig=${WORKDIR}/${ANTREA_AGENT_ANTREA_KUBECONFIG} set-credentials antrea-agent --token=$TOKEN
kubectl config --kubeconfig=${WORKDIR}/${ANTREA_AGENT_ANTREA_KUBECONFIG} set-context antrea-agent@antrea --cluster=antrea --user=antrea-agent
Expand Down
4 changes: 2 additions & 2 deletions docs/external-node.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -208,7 +208,7 @@ spec:
NAMESPACE="vm-ns"
KUBECONFIG="antrea-agent.kubeconfig"
APISERVER=$(kubectl config view -o jsonpath="{.clusters[?(@.name==\"$CLUSTER_NAME\")].cluster.server}")
TOKEN=$(kubectl -n $NAMESPACE get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='$SERVICE_ACCOUNT')].data.token}"|base64 --decode)
TOKEN=$(kubectl -n $NAMESPACE get secrets -o jsonpath="{.items[?(@.metadata.name=='${SERVICE_ACCOUNT}-service-account-token')].data.token}"|base64 --decode)
kubectl config --kubeconfig=$KUBECONFIG set-cluster $CLUSTER_NAME --server=$APISERVER --insecure-skip-tls-verify=true
kubectl config --kubeconfig=$KUBECONFIG set-credentials antrea-agent --token=$TOKEN
kubectl config --kubeconfig=$KUBECONFIG set-context antrea-agent@$CLUSTER_NAME --cluster=$CLUSTER_NAME --user=antrea-agent
Expand All @@ -226,7 +226,7 @@ spec:
ANTREA_CLUSTER_NAME="antrea"
NAMESPACE="vm-ns"
KUBECONFIG="antrea-agent.antrea.kubeconfig"
TOKEN=$(kubectl -n $NAMESPACE get secrets -o jsonpath="{.items[?(@.metadata.annotations['kubernetes\.io/service-account\.name']=='$SERVICE_ACCOUNT')].data.token}"|base64 --decode)
TOKEN=$(kubectl -n $NAMESPACE get secrets -o jsonpath="{.items[?(@.metadata.name=='${SERVICE_ACCOUNT}-service-account-token')].data.token}"|base64 --decode)
kubectl config --kubeconfig=$KUBECONFIG set-cluster $ANTREA_CLUSTER_NAME --server=$ANTREA_API_SERVER --insecure-skip-tls-verify=true
kubectl config --kubeconfig=$KUBECONFIG set-credentials antrea-agent --token=$TOKEN
kubectl config --kubeconfig=$KUBECONFIG set-context antrea-agent@$ANTREA_CLUSTER_NAME --cluster=$ANTREA_CLUSTER_NAME --user=antrea-agent
Expand Down