Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Send pod name/ns to nodeagent #2790

Merged
merged 2 commits into from
Feb 28, 2024
Merged

Send pod name/ns to nodeagent #2790

merged 2 commits into from
Feb 28, 2024

Conversation

jayanthvn
Copy link
Contributor

@jayanthvn jayanthvn commented Feb 7, 2024

What type of PR is this?

Feature

Which issue does this PR fix:
N/A

What does this PR do / Why do we need it:
In certain scenarios we would need to send pod name/ns to node agent

If an issue # is not available please add repro steps and logs from IPAMD/CNI showing the issue:
N/A

Testing done on this change:

Yes

Failed call to NA and kubelet will send delete so need to call cleanup... -

{"level":"info","ts":"2024-02-14T17:28:56.972Z","caller":"routed-eni-cni-plugin/cni.go:196","msg":"Received add network response from ipamd for container 0f281751103d315a94be105eb2c37e7e4f030cbd51dca3d72803306722fbafe5 interface eth0: Success:true IPv4Addr:\"192.168.54.253\" DeviceNumber:2 VPCv4CIDRs:\"192.168.0.0/16\" NetworkPolicyMode:\"strict\" "}
{"level":"debug","ts":"2024-02-14T17:28:56.972Z","caller":"routed-eni-cni-plugin/cni.go:296","msg":"SetupPodNetwork: hostVethName=eni4a5c3305307, contVethName=eth0, netnsPath=/var/run/netns/cni-2a7be20f-ffff-d303-9ea0-783cdf93c4c9, v4Addr=192.168.54.253/32, v6Addr=<nil>, deviceNumber=2, mtu=9001"}
{"level":"debug","ts":"2024-02-14T17:28:57.008Z","caller":"driver/driver.go:237","msg":"Successfully set IPv6 sysctls on hostVeth eni4a5c3305307"}
{"level":"debug","ts":"2024-02-14T17:28:57.008Z","caller":"driver/driver.go:253","msg":"Successfully setup container route, containerAddr=192.168.54.253/32, hostVeth=eni4a5c3305307, rtTable=main"}
{"level":"debug","ts":"2024-02-14T17:28:57.008Z","caller":"driver/driver.go:253","msg":"Successfully setup toContainer rule, containerAddr=192.168.54.253/32, rtTable=main"}
{"level":"debug","ts":"2024-02-14T17:28:57.008Z","caller":"driver/driver.go:253","msg":"Successfully setup fromContainer rule, containerAddr=192.168.54.253/32, rtTable=3"}
{"level":"debug","ts":"2024-02-14T17:28:57.008Z","caller":"routed-eni-cni-plugin/cni.go:196","msg":"Using dummy interface: {Name:dummy4a5c3305307 Mac:0 Sandbox:2}"}
{"level":"error","ts":"2024-02-14T17:28:57.013Z","caller":"routed-eni-cni-plugin/cni.go:196","msg":"Failed to setup default network policy Pod Name websocket-client-7f65cc6544-sbwwz and NameSpace default: GRPC returned - rpc error: code = Unavailable desc = error reading from server: EOF Network policy agent returned - <nil>"}
{"level":"info","ts":"2024-02-14T17:28:57.030Z","caller":"routed-eni-cni-plugin/cni.go:119","msg":"Constructed new logger instance"}
{"level":"debug","ts":"2024-02-14T17:28:57.031Z","caller":"routed-eni-cni-plugin/cni.go:381","msg":"Prev Result: <nil>\n"}
{"level":"info","ts":"2024-02-14T17:28:57.031Z","caller":"routed-eni-cni-plugin/cni.go:381","msg":"Received CNI del request: ContainerID(0f281751103d315a94be105eb2c37e7e4f030cbd51dca3d72803306722fbafe5) Netns(/var/run/netns/cni-2a7be20f-ffff-d303-9ea0-783cdf93c4c9) IfName(eth0) Args(IgnoreUnknown=1;K8S_POD_NAMESPACE=default;K8S_POD_NAME=websocket-client-7f65cc6544-sbwwz;K8S_POD_INFRA_CONTAINER_ID=0f281751103d315a94be105eb2c37e7e4f030cbd51dca3d72803306722fbafe5;K8S_POD_UID=a2385f19-a5e7-44fa-b2ab-9009213c1ea2) Path(/opt/cni/bin) argsStdinData({\"cniVersion\":\"0.4.0\",\"mtu\":\"9001\",\"name\":\"aws-cni\",\"pluginLogFile\":\"/var/log/aws-routed-eni/plugin.log\",\"pluginLogLevel\":\"DEBUG\",\"podSGEnforcingMode\":\"strict\",\"type\":\"aws-cni\",\"vethPrefix\":\"eni\"})"}
{"level":"info","ts":"2024-02-14T17:28:57.033Z","caller":"routed-eni-cni-plugin/cni.go:381","msg":"Received del network response from ipamd for pod websocket-client-7f65cc6544-sbwwz namespace default sandbox 0f281751103d315a94be105eb2c37e7e4f030cbd51dca3d72803306722fbafe5: Success:true IPv4Addr:\"192.168.54.253\" DeviceNumber:2 "}
{"level":"debug","ts":"2024-02-14T17:28:57.033Z","caller":"routed-eni-cni-plugin/cni.go:503","msg":"TeardownPodNetwork: containerAddr=192.168.54.253/32, deviceNumber=2"}
{"level":"debug","ts":"2024-02-14T17:28:57.033Z","caller":"driver/driver.go:267","msg":"Successfully deleted toContainer rule, containerAddr=192.168.54.253/32, rtTable=main"}
{"level":"debug","ts":"2024-02-14T17:28:57.033Z","caller":"driver/driver.go:267","msg":"Successfully deleted fromContainer rule, containerAddr=192.168.54.253/32, rtTable=3"}
{"level":"debug","ts":"2024-02-14T17:28:57.033Z","caller":"driver/driver.go:267","msg":"Successfully deleted container route, containerAddr=192.168.54.253/32, rtTable=main"}

Successful call to NA -

{"level":"info","ts":"2024-02-14T08:23:06.969Z","caller":"routed-eni-cni-plugin/cni.go:196","msg":"Received CNI add request: ContainerID(fc1a62966a5312f7e5349733fbf741db1eb3790f7cfb336223c985d0e8b510ca) Netns(/var/run/netns/cni-7481806d-2e06-7ac7-1a02-7c3485e43a03) IfName(eth0) Args(IgnoreUnknown=1;K8S_POD_NAMESPACE=default;K8S_POD_NAME=websocket-client-7f65cc6544-jfjdh;K8S_POD_INFRA_CONTAINER_ID=fc1a62966a5312f7e5349733fbf741db1eb3790f7cfb336223c985d0e8b510ca;K8S_POD_UID=929314c7-93af-4267-b629-766c05af7ae4) Path(/opt/cni/bin) argsStdinData({\"cniVersion\":\"0.4.0\",\"mtu\":\"9001\",\"name\":\"aws-cni\",\"pluginLogFile\":\"/var/log/aws-routed-eni/plugin.log\",\"pluginLogLevel\":\"DEBUG\",\"podSGEnforcingMode\":\"strict\",\"type\":\"aws-cni\",\"vethPrefix\":\"eni\"})"}
{"level":"debug","ts":"2024-02-14T08:23:06.969Z","caller":"routed-eni-cni-plugin/cni.go:196","msg":"Prev Result: <nil>\n"}
{"level":"debug","ts":"2024-02-14T08:23:06.969Z","caller":"routed-eni-cni-plugin/cni.go:196","msg":"MTU value set is 9001:"}
{"level":"info","ts":"2024-02-14T08:23:06.973Z","caller":"routed-eni-cni-plugin/cni.go:196","msg":"Received add network response from ipamd for container fc1a62966a5312f7e5349733fbf741db1eb3790f7cfb336223c985d0e8b510ca interface eth0: Success:true IPv4Addr:\"192.168.58.56\" DeviceNumber:1 VPCv4CIDRs:\"192.168.0.0/16\" NetworkPolicyMode:\"strict\" "}
{"level":"debug","ts":"2024-02-14T08:23:06.973Z","caller":"routed-eni-cni-plugin/cni.go:296","msg":"SetupPodNetwork: hostVethName=eniabdd087b487, contVethName=eth0, netnsPath=/var/run/netns/cni-7481806d-2e06-7ac7-1a02-7c3485e43a03, v4Addr=192.168.58.56/32, v6Addr=<nil>, deviceNumber=1, mtu=9001"}
{"level":"debug","ts":"2024-02-14T08:23:07.020Z","caller":"driver/driver.go:237","msg":"Successfully set IPv6 sysctls on hostVeth eniabdd087b487"}
{"level":"debug","ts":"2024-02-14T08:23:07.020Z","caller":"driver/driver.go:253","msg":"Successfully setup container route, containerAddr=192.168.58.56/32, hostVeth=eniabdd087b487, rtTable=main"}
{"level":"debug","ts":"2024-02-14T08:23:07.020Z","caller":"driver/driver.go:253","msg":"Successfully setup toContainer rule, containerAddr=192.168.58.56/32, rtTable=main"}
{"level":"debug","ts":"2024-02-14T08:23:07.020Z","caller":"driver/driver.go:253","msg":"Successfully setup fromContainer rule, containerAddr=192.168.58.56/32, rtTable=2"}
{"level":"debug","ts":"2024-02-14T08:23:07.020Z","caller":"routed-eni-cni-plugin/cni.go:196","msg":"Using dummy interface: {Name:dummyabdd087b487 Mac:0 Sandbox:1}"}
{"level":"debug","ts":"2024-02-14T08:23:07.128Z","caller":"routed-eni-cni-plugin/cni.go:196","msg":"Network Policy agent returned Success : true"}

Enabling strict mode even coredns will be blocked and egress from client pod will be blocked-

{"level":"info","ts":"2024-02-14T08:26:49.020Z","logger":"ebpf-client","msg":"Flow Info:  ","Src IP":"192.168.58.56","Src Port":34043,"Dest IP":"10.100.0.10","Dest Port":53,"Proto":"UDP","Verdict":"DENY"}
{"level":"info","ts":"2024-02-14T08:26:49.020Z","logger":"ebpf-client","msg":"Flow Info:  ","Src IP":"192.168.58.56","Src Port":48442,"Dest IP":"10.100.0.10","Dest Port":53,"Proto":"UDP","Verdict":"DENY"}

Upon enabling egress on client pod and ingress on coredns pods -

"Verdict":"ACCEPT"}
{"level":"info","ts":"2024-02-14T08:33:31.010Z","logger":"ebpf-client","msg":"Flow Info:  ","Src IP":"192.168.58.56","Src Port":58073,"Dest IP":"10.100.0.10","Dest Port":53,"Proto":"UDP","Verdict":"ACCEPT"}
{"level":"info","ts":"2024-02-14T08:33:31.010Z","logger":"ebpf-client","msg":"Flow Info:  ","Src IP":"192.168.58.56","Src Port":43721,"Dest IP":"10.100.0.10","Dest Port":53,"Proto":"UDP","Verdict":"ACCEPT"}
{"level":"info","ts":"2024-02-14T08:33:31.011Z","logger":"ebpf-client","msg":"Flow Info:  ","Src IP":"192.168.58.56","Src Port":33020,"Dest IP":"10.100.58.78","Dest Port":80,"Proto":"TCP","Verdict":"ACCEPT"}

But client to server is blocked, since server hasn't allowed ingress -

"level":"info","ts":"2024-02-14T08:34:02.540Z","logger":"ebpf-client","msg":"Flow Info:  ","Src IP":"192.168.58.56","Src Port":33020,"Dest IP":"192.168.50.232","Dest Port":8080,"Proto":"TCP","Verdict":"DENY"} 

default       websocket-client-7f65cc6544-jfjdh                       0/1     CrashLoopBackOff   5 (72s ago)    13m     192.168.58.56    ip-192-168-50-132.us-west-2.compute.internal   <none>           <none>
default       websocket-server-64ddc8864-zrl9v                        1/1     Running            0              6m46s   192.168.50.232   ip-192-168-50-132.us-west-2.compute.internal   <none>           <none>

Upon allowing client to server -

{"level":"info","ts":"2024-02-14T08:38:06.864Z","logger":"ebpf-client","msg":"Flow Info:  ","Src IP":"192.168.52.244","Src Port":37262,"Dest IP":"192.168.50.232","Dest Port":8080,"Proto":"TCP","Verdict":"ACCEPT"}

Even pod will be running -

default       websocket-client-7f65cc6544-xrt2z                       1/1     Running   0              20s     192.168.52.244   ip-192-168-50-132.us-west-2.compute.internal   <none>           <none>
default       websocket-server-64ddc8864-zrl9v                        1/1     Running   0              13m     192.168.50.232   ip-192-168-50-132.us-west-2.compute.internal   <none>           <none>

Will this PR introduce any new dependencies?:

No

Will this break upgrades or downgrades? Has updating a running cluster been tested?:
No

Does this change require updates to the CNI daemonset config files to work?:

No

Does this PR introduce any user-facing change?:

No

New env variable - NETWORK_POLICY_ENFORCING_MODE:   [none, strict, standard]

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@jayanthvn jayanthvn requested a review from a team as a code owner February 7, 2024 22:09
@jayanthvn jayanthvn marked this pull request as draft February 7, 2024 22:09
@jayanthvn jayanthvn requested a review from achevuru February 7, 2024 22:11
cmd/routed-eni-cni-plugin/cni.go Outdated Show resolved Hide resolved
cmd/routed-eni-cni-plugin/cni.go Show resolved Hide resolved
cmd/routed-eni-cni-plugin/cni_test.go Outdated Show resolved Hide resolved
cmd/routed-eni-cni-plugin/cni_test.go Show resolved Hide resolved
pkg/ipamd/ipamd.go Show resolved Hide resolved
@jayanthvn jayanthvn marked this pull request as ready for review February 14, 2024 08:40
pkg/ipamd/ipamd.go Outdated Show resolved Hide resolved
pkg/ipamd/ipamd.go Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants