Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Thiea Manager with basic functionality #97

Merged
merged 3 commits into from
Sep 10, 2022

Conversation

wsquan171
Copy link
Contributor

@wsquan171 wsquan171 commented Aug 24, 2022

This change adds the skeleton of Theia manager, including APIServer and a simple NetworkPolicyRecommendation CR controller.

The code is structured following Antrea controller, and summarized below:

  • pkg/apis/crd structured CRD definitions for k8s client generation (consumed by controllers)
  • pkg/apis/intelligence structured REST payload definitions by API group and endpoints
  • pkg/apiserver/registry REST handlers grouped by API group and endpoints
  • pkg/apiserver/apiserver.go APIServer definitions and configurations
  • pkg/client k8s client-gen'ed codes
  • pkg/controller collection of k8s resource controllers

This change also adds Helm chart generation and code generation scripts.

To get the manager up and running in an existing testbed:

  1. Apply build/charts/theia/crds/network-policy-recommendation-crd.yaml
  2. make theia-manager and scp image projects.registry.vmware.com/antrea/theia-manager:latest to worker nodes
  3. hack/generate-manifest.sh --mode dev --theia-manager > build/yamls/flow-visibility.yml and apply the diffs

With the following CR deployed:

apiVersion: crd.theia.antrea.io/v1alpha1
kind: NetworkPolicyRecommendation
metadata:
  name: pr-test
spec:
  jobType: "initial"
  limit: 1000
  policyType: "anp-deny-all"
  startTime: 2022-08-24T10:45:57Z

$ curl -k https://<theia-manager-svc>/apis/intelligence.theia.antrea.io/v1alpha1/networkpolicyrecommendations/pr-test

{
  "kind": "NetworkPolicyRecommendation",
  "apiVersion": "intelligence.theia.antrea.io/v1alpha1",
  "metadata": {
    "name": "pr-test",
    "creationTimestamp": null
  },
  "limit": 1000,
  "policyType": "anp-deny-all",
  "startTime": "2022-08-24T10:45:57Z",
  "endTime": null
}

curl -k https://<theia-manager-svc>apis/intelligence.theia.antrea.io/v1alpha1/networkpolicyrecommendations/pr-test-1

{
  "kind": "Status",
  "apiVersion": "v1",
  "metadata": {},
  "status": "Failure",
  "message": "networkpolicyrecommendations.intelligence.theia.antrea.io \"pr-test-1\" not found",
  "reason": "NotFound",
  "details": {
    "name": "pr-test-1",
    "group": "intelligence.theia.antrea.io",
    "kind": "networkpolicyrecommendations"
  },
  "code": 404
}

build/charts/theia/README.md Outdated Show resolved Hide resolved
build/charts/theia/conf/theia-manager.conf Outdated Show resolved Hide resolved
build/charts/theia/values.yaml Outdated Show resolved Hide resolved
pkg/config/theiamanager/config.go Outdated Show resolved Hide resolved
@wsquan171 wsquan171 force-pushed the manager branch 3 times, most recently from 7e42527 to ba3ae5f Compare August 25, 2022 18:16
pkg/apiserver/apiserver.go Outdated Show resolved Hide resolved
cmd/theia-manager/options.go Outdated Show resolved Hide resolved
Copy link
Contributor

@yanjunz97 yanjunz97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks Shawn for working on this!

build/images/Dockerfile.theia-manager.ubuntu Outdated Show resolved Hide resolved
cmd/theia-manager/theia-manager.go Outdated Show resolved Hide resolved
cmd/theia-manager/theia-manager.go Outdated Show resolved Hide resolved
cmd/theia-manager/theia-manager.go Outdated Show resolved Hide resolved
Namespace: npReco.Namespace,
Name: npReco.Name,
}
c.queue.Add(namespacedName)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should it be c.queue.forget(namespacedName)?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

c.queue is the actual worker queue to reconcile the resources. here we're just adding the key to the workqueue to indicate that some resource is removed and reconciliation is needed (i.e. deletion of child res related to this job CR). the key is forgotten only when reconciliation is done for it.

@wsquan171 wsquan171 force-pushed the manager branch 2 times, most recently from c03db5d to 829a288 Compare August 26, 2022 17:24
const informerDefaultResync = 12 * time.Hour

func run(o *Options) error {
klog.Infof("Theia manager starting...")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

better to use structured logging: klog.InfoS(). Same for other logs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed 4 occurrences

pkg/apis/crd/v1alpha1/types.go Show resolved Hide resolved
pkg/apis/crd/v1alpha1/types.go Show resolved Hide resolved
pkg/apis/intelligence/v1alpha1/types.go Show resolved Hide resolved
}
p := s.GenericAPIServer
if p == nil {
return fmt.Errorf("nil")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed both. was leftover when doing local debugging. neither should be the case when actually running the code.

pkg/apiserver/apiserver.go Outdated Show resolved Hide resolved
@wsquan171 wsquan171 force-pushed the manager branch 2 times, most recently from 7e737bd to 9fde85d Compare September 2, 2022 18:49
This change adds the following:
1. API server setup and config
2. sample REST endpoint apis/intelligence.theia.antrea.io/v1alpha1/networkpolicyrecommendations
3. codegen scripts for API server
4. helm charts for theia Manager and API server config values

Signed-off-by: Shawn Wang <wshaoquan@vmware.com>
This change adds an example controller to Theia manager, which watches
NetworkPolicyRecommendation and exposes querier interface for APIServer
to consume.

When NetworkPolicyRecommendation is deployed in flow-visibility NS,
the k8s resources and be properly returned via REST endpoint
/apis/intelligence.theia.antrea.io/v1alpha1/networkpolicyrecommendations/{name}

Signed-off-by: Shawn Wang <wshaoquan@vmware.com>
Signed-off-by: Shawn Wang <wshaoquan@vmware.com>
Copy link
Contributor

@yanjunz97 yanjunz97 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@salv-orlando salv-orlando left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@wsquan171 wsquan171 merged commit 486e586 into antrea-io:main Sep 10, 2022
@wsquan171 wsquan171 deleted the manager branch September 10, 2022 00:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants