Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Versioned API and Antctl for NetworkPolicyEvaluation (effective policy rule prediction) #5740

Merged
merged 1 commit into from
Mar 4, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,10 +106,10 @@ Also check out [@ProjectAntrea](https://twitter.com/ProjectAntrea) on Twitter!
enable fine-grained visibility into the communication among Kubernetes
workloads. Theia provides visualization for Antrea network flows in Grafana
dashboards, and recommends Network Policies to secure the workloads.
* **Network Policies for virtual machines**: Antrea native policies can be
* **Network Policies for virtual machines**: Antrea-native policies can be
enforced on non-Kubernetes Nodes including VMs and baremetal servers. Project
[Nephe](https://github.com/antrea-io/nephe) implements security policies for
VMs across clouds, leveraging Antrea native policies.
VMs across clouds, leveraging Antrea-native policies.
* **Encryption**: Encryption of inter-Node Pod traffic with IPsec or WireGuard
tunnels.
* **Easy deployment**: Antrea is deployed by applying a single YAML manifest
Expand Down
6 changes: 6 additions & 0 deletions build/charts/antrea/templates/antctl/clusterrole.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,12 @@ rules:
verbs:
- get
- list
- apiGroups:
- controlplane.antrea.io
resources:
- networkpolicyevaluation
verbs:
- create
qiyueyao marked this conversation as resolved.
Show resolved Hide resolved
- apiGroups:
- stats.antrea.io
resources:
Expand Down
6 changes: 6 additions & 0 deletions build/yamls/antrea-aks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6339,6 +6339,12 @@ rules:
verbs:
- get
- list
- apiGroups:
- controlplane.antrea.io
resources:
- networkpolicyevaluation
verbs:
- create
- apiGroups:
- stats.antrea.io
resources:
Expand Down
6 changes: 6 additions & 0 deletions build/yamls/antrea-eks.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6339,6 +6339,12 @@ rules:
verbs:
- get
- list
- apiGroups:
- controlplane.antrea.io
resources:
- networkpolicyevaluation
verbs:
- create
- apiGroups:
- stats.antrea.io
resources:
Expand Down
6 changes: 6 additions & 0 deletions build/yamls/antrea-gke.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6339,6 +6339,12 @@ rules:
verbs:
- get
- list
- apiGroups:
- controlplane.antrea.io
resources:
- networkpolicyevaluation
verbs:
- create
- apiGroups:
- stats.antrea.io
resources:
Expand Down
6 changes: 6 additions & 0 deletions build/yamls/antrea-ipsec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6352,6 +6352,12 @@ rules:
verbs:
- get
- list
- apiGroups:
- controlplane.antrea.io
resources:
- networkpolicyevaluation
verbs:
- create
- apiGroups:
- stats.antrea.io
resources:
Expand Down
6 changes: 6 additions & 0 deletions build/yamls/antrea.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6339,6 +6339,12 @@ rules:
verbs:
- get
- list
- apiGroups:
- controlplane.antrea.io
resources:
- networkpolicyevaluation
verbs:
- create
- apiGroups:
- stats.antrea.io
resources:
Expand Down
15 changes: 15 additions & 0 deletions docs/antctl.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@ running in three different modes:
- [controllerinfo and agentinfo commands](#controllerinfo-and-agentinfo-commands)
- [NetworkPolicy commands](#networkpolicy-commands)
- [Mapping endpoints to NetworkPolicies](#mapping-endpoints-to-networkpolicies)
- [Evaluating expected NetworkPolicy behavior](#evaluating-expected-networkpolicy-behavior)
- [Dumping Pod network interface information](#dumping-pod-network-interface-information)
- [Dumping OVS flows](#dumping-ovs-flows)
- [OVS packet tracing](#ovs-packet-tracing)
Expand Down Expand Up @@ -263,6 +264,20 @@ Namespace.
This command only works in "controller mode" and **as of now it can only be run
from inside the Antrea Controller Pod, and not from out-of-cluster**.

#### Evaluating expected NetworkPolicy behavior

`antctl` supports evaluating all the existing Antrea-native NetworkPolicies,
Kubernetes NetworkPolicies and AdminNetworkPolicies to predict the effective
policy rule for traffic between source and destination Pods.

```bash
antctl query networkpolicyevaluation -S NAMESPACE/POD -D NAMESPACE/POD
```

If only Pod name is provided, the command will default to the "default" Namespace.

This command only works in "controller mode".

### Dumping Pod network interface information

`antctl` agent command `get podinterface` (or `get pi`) can dump network
Expand Down
2 changes: 1 addition & 1 deletion docs/feature-gates.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ This feature is currently only supported for Nodes running Linux. Windows suppor
Stats API, which can be accessed by kubectl get commands, e.g. `kubectl get networkpolicystats`. The statistical data
includes total number of sessions, packets, and bytes allowed or denied by a NetworkPolicy. It is collected
asynchronously so there may be a delay of up to 1 minute for changes to be reflected in API responses. The feature
supports K8s NetworkPolicies and Antrea native policies, the latter of which requires
supports K8s NetworkPolicies and Antrea-native policies, the latter of which requires
`AntreaPolicy` to be enabled. Usage examples:

```bash
Expand Down
2 changes: 1 addition & 1 deletion docs/multicluster/user-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@

Antrea Multi-cluster implements [Multi-cluster Service API](https://github.com/kubernetes/enhancements/tree/master/keps/sig-multicluster/1645-multi-cluster-services-api),
which allows users to create multi-cluster Services that can be accessed cross
clusters in a ClusterSet. Antrea Multi-cluster also extends Antrea native
clusters in a ClusterSet. Antrea Multi-cluster also extends Antrea-native
NetworkPolicy to support Multi-cluster NetworkPolicy rules that apply to
cross-cluster traffic, and ClusterNetworkPolicy replication that allows a
ClusterSet admin to create ClusterNetworkPolicies which are replicated across
Expand Down
2 changes: 1 addition & 1 deletion docs/traceflow-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -140,7 +140,7 @@ will fail. But you can specify a different timeout value, by adding
`timeout: <value-in-seconds>` to the Traceflow `spec`.

In some cases, it might be useful to capture the packets dropped by
NetworkPolicies (inc. K8s NetworkPolicies or Antrea native policies). You can
NetworkPolicies (inc. K8s NetworkPolicies or Antrea-native policies). You can
add `droppedOnly: true` to the live-traffic Traceflow `spec`, then the first
packet that matches the Traceflow spec and is dropped by a NetworkPolicy will
be captured and traced.
Expand Down
2 changes: 1 addition & 1 deletion hack/update-codegen-dockerized.sh
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ MOCKGEN_TARGETS=(
"pkg/agent/util/netlink Interface testing mock_netlink_linux.go"
"pkg/agent/wireguard Interface testing mock_wireguard.go"
"pkg/antctl AntctlClient ."
"pkg/controller/networkpolicy EndpointQuerier testing"
"pkg/controller/networkpolicy EndpointQuerier,PolicyRuleQuerier testing"
"pkg/controller/querier ControllerQuerier testing"
"pkg/flowaggregator/exporter Interface testing"
"pkg/ipfix IPFIXExportingProcess,IPFIXRegistry,IPFIXCollectingProcess,IPFIXAggregationProcess testing"
Expand Down
2 changes: 1 addition & 1 deletion pkg/agent/controller/networkpolicy/audit_logging.go
Original file line number Diff line number Diff line change
Expand Up @@ -281,7 +281,7 @@ func getNetworkPolicyInfo(pktIn *ofctrl.PacketIn, packet *binding.Packet, c *Con
ob.ofPriority = ofPriority
ob.ruleName = ruleName
ob.logLabel = logLabel
// Fill in placeholders for Antrea native policies without log labels,
// Fill in placeholders for Antrea-native policies without log labels,
// K8s NetworkPolicies without rule names or log labels.
fillLogInfoPlaceholders([]*string{&ob.ruleName, &ob.logLabel, &ob.ofPriority})
return nil
Expand Down
32 changes: 32 additions & 0 deletions pkg/antctl/antctl.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (
"antrea.io/antrea/pkg/agent/apiserver/handlers/podinterface"
"antrea.io/antrea/pkg/agent/apiserver/handlers/serviceexternalip"
fallbackversion "antrea.io/antrea/pkg/antctl/fallback/version"
"antrea.io/antrea/pkg/antctl/parameter"
"antrea.io/antrea/pkg/antctl/raw/featuregates"
"antrea.io/antrea/pkg/antctl/raw/multicluster"
"antrea.io/antrea/pkg/antctl/raw/proxy"
Expand Down Expand Up @@ -512,6 +513,37 @@ $ antctl get podmulticaststats pod -n namespace`,
},
transformedResponse: reflect.TypeOf(endpointserver.EndpointQueryResponse{}),
},
{
use: "networkpolicyevaluation",
aliases: []string{"networkpoliciesevaluation", "networkpolicyeval", "networkpolicieseval", "netpoleval"},
short: "Analyze effective NetworkPolicy rules.",
long: "Analyze network policies in the cluster and return the rule expected to be effective on the source and destination endpoints provided.",
example: ` Query effective NetworkPolicy rule between two Pods
$ antctl query networkpolicyevaluation -S ns1/pod1 -D ns2/pod2
`,
commandGroup: query,
controllerEndpoint: &endpoint{
resourceEndpoint: &resourceEndpoint{
groupVersionResource: &cpv1beta.NetworkPolicyEvaluationVersionResource,
params: []flagInfo{
{
name: "source",
usage: "Source endpoint, specified by <Namespace>/<name>.",
shorthand: "S",
},
{
name: "destination",
usage: "Destination endpoint, specified by <Namespace>/<name>.",
shorthand: "D",
},
},
parameterTransform: parameter.NewNetworkPolicyEvaluation,
restMethod: restPost,
},
addonTransform: networkpolicy.EvaluationTransform,
},
transformedResponse: reflect.TypeOf(networkpolicy.EvaluationResponse{}),
},
{
use: "flowrecords",
short: "Print the matching flow records in the flow aggregator",
Expand Down
26 changes: 21 additions & 5 deletions pkg/antctl/client.go
Original file line number Diff line number Diff line change
Expand Up @@ -161,22 +161,38 @@ func (c *client) resourceRequest(e *resourceEndpoint, opt *requestOption) (io.Re
// If timeout is zero, there will be no timeout.
restClient.Client.Timeout = opt.timeout

resGetter := restClient.Get().
var restRequest *rest.Request
if e.restMethod == restGet {
restRequest = restClient.Get()
} else if e.restMethod == restPost {
restRequest = restClient.Post()
}

restRequest = restRequest.
NamespaceIfScoped(opt.args["namespace"], e.namespaced).
Resource(e.groupVersionResource.Resource)

if len(e.resourceName) != 0 {
resGetter = resGetter.Name(e.resourceName)
restRequest = restRequest.Name(e.resourceName)
} else if name, ok := opt.args["name"]; ok {
resGetter = resGetter.Name(name)
restRequest = restRequest.Name(name)
}

for arg, val := range opt.args {
if arg != "name" && arg != "namespace" {
resGetter = resGetter.Param(arg, val)
restRequest = restRequest.Param(arg, val)
}
}
result := resGetter.Do(context.TODO())

if e.parameterTransform != nil {
obj, err := e.parameterTransform(opt.args)
if err != nil {
return nil, err
}
restRequest = restRequest.Body(obj)
}

result := restRequest.Do(context.TODO())
if result.Error() != nil {
return nil, generateMessage(opt.commandDefinition, opt.args, true /* isResourceRequest */, result.Error())
}
Expand Down
14 changes: 13 additions & 1 deletion pkg/antctl/command_definition.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import (

"github.com/spf13/cobra"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
k8sruntime "k8s.io/apimachinery/pkg/runtime"
"k8s.io/apimachinery/pkg/runtime/schema"
"k8s.io/klog/v2"

Expand Down Expand Up @@ -104,6 +105,9 @@ type resourceEndpoint struct {
resourceName string
namespaced bool
supportSorting bool
params []flagInfo
parameterTransform func(args map[string]string) (k8sruntime.Object, error)
restMethod restMethod
}

func (e *resourceEndpoint) OutputType() OutputType {
Expand Down Expand Up @@ -134,6 +138,7 @@ func (e *resourceEndpoint) flags() []flagInfo {
if e.supportSorting {
flags = append(flags, getSortByFlag())
}
flags = append(flags, e.params...)
return flags
}

Expand All @@ -145,6 +150,13 @@ func getSortByFlag() flagInfo {
}
}

type restMethod uint

const (
restGet restMethod = iota
restPost
)

type nonResourceEndpoint struct {
path string
params []flagInfo
Expand Down Expand Up @@ -434,6 +446,7 @@ func (cd *commandDefinition) output(resp io.Reader, writer io.Writer, ft formatt
if cd.controllerEndpoint.nonResourceEndpoint != nil && cd.controllerEndpoint.nonResourceEndpoint.path == "/endpoint" {
return output.TableOutputForQueryEndpoint(obj, writer)
}
return output.TableOutputForGetCommands(obj, writer)
} else {
return output.TableOutput(obj, writer)
}
Expand All @@ -442,7 +455,6 @@ func (cd *commandDefinition) output(resp io.Reader, writer io.Writer, ft formatt
default:
return fmt.Errorf("unsupported format type: %v", ft)
}
return nil
}

func (cd *commandDefinition) collectFlags(cmd *cobra.Command, args []string) (map[string]string, error) {
Expand Down
57 changes: 57 additions & 0 deletions pkg/antctl/parameter/parameter.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
// Copyright 2024 Antrea Authors
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

package parameter

import (
"fmt"
"strings"

"k8s.io/apimachinery/pkg/runtime"

cpv1beta "antrea.io/antrea/pkg/apis/controlplane/v1beta2"
)

// parsePeer parses Namespace/Pod name, empty string is returned if the argument is not of a
// valid Namespace/Pod reference (missing pod name or invalid format). Namespace will be set
// as default if missing, string without separator will be considered as pod name.
func parsePeer(str string) (string, string) {
parts := strings.Split(str, "/")
ns, pod := "", ""
if len(parts) == 1 {
ns, pod = "default", parts[0]
} else if len(parts) == 2 {
ns, pod = parts[0], parts[1]
}
return ns, pod
}

func NewNetworkPolicyEvaluation(args map[string]string) (runtime.Object, error) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should it be moved to transform/networkpolicy like the type of the response? otherwise this file would include all parameters of discrete commands.

Copy link
Contributor Author

@qiyueyao qiyueyao Feb 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The function was moved here as a new package parameter based on this comment, if I understood it correctly.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should probably find a better package name / location in a future PR though

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, do you have any suggestion or insight? To include it inside transform or refactor some of the existing structs?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thinking about it some more, maybe we could keep it in transform/networkpolicy, but as a separate file to clearly mark the difference? Maybe we could have transform/networkpolicy/request.go and ransform/networkpolicy/response.go?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, will open a PR for this.

var ns1, pod1, ns2, pod2 string
if val, ok := args["source"]; ok {
ns1, pod1 = parsePeer(val)
}
if val, ok := args["destination"]; ok {
ns2, pod2 = parsePeer(val)
}
if pod1 == "" || pod2 == "" {
return nil, fmt.Errorf("missing entities for NetworkPolicyEvaluation request: %v", args)
}
return &cpv1beta.NetworkPolicyEvaluation{
Request: &cpv1beta.NetworkPolicyEvaluationRequest{
Source: cpv1beta.Entity{Pod: &cpv1beta.PodReference{Namespace: ns1, Name: pod1}},
Destination: cpv1beta.Entity{Pod: &cpv1beta.PodReference{Namespace: ns2, Name: pod2}},
},
}, nil
}
Loading
Loading