add e2e test for security group for pods

[abhipth]

Verifies the following

• Branch ENI -> Regular Pod TCP Connection
• Branch ENI -> Branch ENI Pod TCP Connection
• Liveness probe doesn't fail on disabling early TCP Socket demux
• Liveness probe fails when TCP Socket demux is enabled

What type of PR is this?
E2E Test

Which issue does this PR fix:
NA

What does this PR do / Why do we need it:
Adds e2e test

Testing done on this change:

Running Suite: Security Group for Pods e2e Test Suite
=====================================================
Random Seed: 1620378193
Will run 5 of 5 specs

STEP: creating ec2 key-pair for the new node group
STEP: creating a new security group used in Security Group Policy
STEP: authorizing egress and ingress on security group for 
STEP: getting the cluster VPC Config
STEP: getting the cluster role name
STEP: attaching the AmazonEKSVPCResourceController policy from the cluster role
STEP: creating a new self managed node group
STEP: enabling pod eni on aws-node DaemonSet
STEP: getting the aws-node daemon set in namesapce kube-system
STEP: setting the environment variables on the ds to map[ENABLE_POD_ENI:true]
STEP: updating the daemon set with new environment variable
Security Group for Pods Test when testing traffic between branch ENI pods and regular pods 
  should have 99%+ success rate
  /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:84
STEP: creating test namespace
STEP: creating the Security Group Policy
successfully created server deployment
successfully created metric server pod
successfully fetched list of pod belonging to deployment
STEP: validating pod traffic-server-6948dd5fbb-n4xjg has branch ENI eni-0f6d3488fd63d8f6c
STEP: validating pod traffic-server-6948dd5fbb-5h8pn has branch ENI eni-0ee56f5237b62023c
STEP: validating pod traffic-server-6948dd5fbb-l68xv has branch ENI eni-02c9845934455d982
STEP: validating pod traffic-server-6948dd5fbb-dgx29 has branch ENI eni-0f88eb89c82c18e6e
STEP: validating pod traffic-server-6948dd5fbb-7j2b8 has branch ENI eni-0d6482fdf6967f0f4
STEP: validating pod traffic-server-6948dd5fbb-zhxb5 has branch ENI eni-0b489f49a5dd5ea47
STEP: validating pod traffic-server-6948dd5fbb-pffm9 has branch ENI eni-004dd323b6c664184
STEP: validating pod traffic-server-6948dd5fbb-bcx2f has branch ENI eni-06cb47b66f0e396df
STEP: validating pod traffic-server-6948dd5fbb-qfvwr has branch ENI eni-0eee8190a66e23db1
STEP: validating pod traffic-server-6948dd5fbb-6kx8q has branch ENI eni-08da141752319d832
STEP: validating pod traffic-server-6948dd5fbb-jq4tb has branch ENI eni-07e8bc26abe2a5d66
STEP: validating pod traffic-server-6948dd5fbb-m8296 has branch ENI eni-0a46bddcf3ee5c7b5
STEP: validating pod traffic-server-6948dd5fbb-pzk62 has branch ENI eni-0323bdf588a9aa026
STEP: validating pod traffic-server-6948dd5fbb-hs7p6 has branch ENI eni-04c0f2669608d27d4
STEP: validating pod traffic-server-6948dd5fbb-4vm78 has branch ENI eni-0d028f364a3c08c75
STEP: validating pod traffic-server-6948dd5fbb-cjvjh has branch ENI eni-0b5b47e51b26bb061
STEP: validating pod traffic-server-6948dd5fbb-4gnnm has branch ENI eni-075fb25a9a9aaf992
STEP: validating pod traffic-server-6948dd5fbb-c6z4w has branch ENI eni-05293980ef993899a
STEP: validating pod traffic-server-6948dd5fbb-qjfv5 has branch ENI eni-0de8c23ad3cc5fe0e
STEP: validating pod traffic-server-6948dd5fbb-mchc5 has branch ENI eni-04a739ed48c5ef1d3
STEP: validating pod traffic-server-6948dd5fbb-4m5gw has branch ENI eni-0e09809ee6c6dcd0b
STEP: validating pod traffic-server-6948dd5fbb-7gsjf has branch ENI eni-0a922cd6092bb4b39
STEP: validating pod traffic-server-6948dd5fbb-rwtj7 has branch ENI eni-036d0f5de9724c62e
STEP: validating pod traffic-server-6948dd5fbb-jnjnh has branch ENI eni-07f006a206da99d3a
STEP: validating pod traffic-server-6948dd5fbb-fl2dz has branch ENI eni-009ad7ce587e61353
STEP: validating pod traffic-server-6948dd5fbb-zqj4c has branch ENI eni-0d8153d2bc51e2a59
STEP: validating pod traffic-server-6948dd5fbb-x5sf9 has branch ENI eni-086c1b62ef13083eb
STEP: validating pod traffic-server-6948dd5fbb-ssjs4 has branch ENI eni-0b8ee76b24c0ac1f7
STEP: validating pod traffic-server-6948dd5fbb-w9cfw has branch ENI eni-096e534ccf1f4c543
STEP: validating pod traffic-server-6948dd5fbb-zqszk has branch ENI eni-0f4f22233832d5d1d
STEP: validating pod traffic-server-6948dd5fbb-6m9nr has branch ENI eni-0598b8f409ee65bda
STEP: validating pod traffic-server-6948dd5fbb-b7k8x has branch ENI eni-06412d7c0af1fa6df
STEP: validating pod traffic-server-6948dd5fbb-gs2nl has branch ENI eni-04ebc9178eaad31d5
STEP: validating pod traffic-server-6948dd5fbb-lrhrd has branch ENI eni-03a59e5aadaa6f88b
STEP: validating pod traffic-server-6948dd5fbb-prnr5 has branch ENI eni-0b4000aea1513ad19
STEP: validating pod traffic-server-6948dd5fbb-vrgxr has branch ENI eni-07a4975e3498cf385
STEP: validating pod traffic-server-6948dd5fbb-72gdd has branch ENI eni-0a751f6d06d4073d1
STEP: validating pod traffic-server-6948dd5fbb-xvgpp has branch ENI eni-0dbae433b9dc0da9e
STEP: validating pod traffic-server-6948dd5fbb-wjbgj has branch ENI eni-04a4c7e655101ea8d
STEP: validating pod traffic-server-6948dd5fbb-fl9z6 has branch ENI eni-0e9ad54cf50b8bb4f
STEP: validating pod traffic-server-6948dd5fbb-c49l7 has branch ENI eni-0eb04e95f59c1a02f
STEP: validating pod traffic-server-6948dd5fbb-t9fxc has branch ENI eni-04ff995416622d533
STEP: validating pod traffic-server-6948dd5fbb-25qdf has branch ENI eni-0f9862b62f102dcbf
STEP: validating pod traffic-server-6948dd5fbb-vtvl4 has branch ENI eni-0029fed99bd1f693d
STEP: validating pod traffic-server-6948dd5fbb-2mcqf has branch ENI eni-0b6cfebe04d65b982
STEP: validating pod traffic-server-6948dd5fbb-gtsvx has branch ENI eni-0780e36e10bb3ca78
STEP: validating pod traffic-server-6948dd5fbb-5jbf9 has branch ENI eni-0882ecb9ca03d6651
STEP: validating pod traffic-server-6948dd5fbb-r2vbd has branch ENI eni-0e42e042365704c66
STEP: validating pod traffic-server-6948dd5fbb-fr9zz has branch ENI eni-0437634ee05c4cb9a
STEP: validating pod traffic-server-6948dd5fbb-8lv2w has branch ENI eni-042fd981f3e2f809c
STEP: validating pod traffic-server-6948dd5fbb-sq5j7 has branch ENI eni-03f2fb3132f8df928
STEP: validating pod traffic-server-6948dd5fbb-8vrnm has branch ENI eni-02090a2d7ed910e27
STEP: validating pod traffic-server-6948dd5fbb-c2n24 has branch ENI eni-09cd058c30fc12254
STEP: validating pod traffic-server-6948dd5fbb-lqnsp has branch ENI eni-02ef89e9fd54a294e
successfully validated the server pod list
successfully created traffic client
successfully fetched the metrics from metric server
SuccessRate: 100STEP: deleting test namespace
STEP: waiting for the branch ENI to be cooled down

• [SLOW TEST:125.517 seconds]
Security Group for Pods Test
/Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:34
  when testing traffic between branch ENI pods and regular pods
  /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:79
    should have 99%+ success rate
    /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:84
------------------------------
Security Group for Pods Test when testing traffic between branch ENI and branch ENI pods 
  should have 99%+ success rate
  /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:110
STEP: creating test namespace
STEP: creating the Security Group Policy
successfully created server deployment
successfully created metric server pod
successfully fetched list of pod belonging to deployment
STEP: validating pod traffic-server-6948dd5fbb-tcfxw has branch ENI eni-0fe9c4969633fa134
STEP: validating pod traffic-server-6948dd5fbb-w7cs4 has branch ENI eni-0373ebf1fed2ea0d2
STEP: validating pod traffic-server-6948dd5fbb-bsqpm has branch ENI eni-031a06441488e120b
STEP: validating pod traffic-server-6948dd5fbb-zbsqz has branch ENI eni-0811f5aa8c407e15b
STEP: validating pod traffic-server-6948dd5fbb-l2v62 has branch ENI eni-072267e7c5bd64ef4
STEP: validating pod traffic-server-6948dd5fbb-t8mxc has branch ENI eni-08f7bbe6fc000d8b2
STEP: validating pod traffic-server-6948dd5fbb-dvszt has branch ENI eni-0eb4ca7ede97959e1
STEP: validating pod traffic-server-6948dd5fbb-kr9nc has branch ENI eni-06b3016826f8bee16
STEP: validating pod traffic-server-6948dd5fbb-ss4n6 has branch ENI eni-05d790120fbef01b0
STEP: validating pod traffic-server-6948dd5fbb-sbpcx has branch ENI eni-095e9ebc90e231b38
STEP: validating pod traffic-server-6948dd5fbb-kfzqq has branch ENI eni-033e9b278eb027da5
STEP: validating pod traffic-server-6948dd5fbb-msxv6 has branch ENI eni-050905f70dbe670e0
STEP: validating pod traffic-server-6948dd5fbb-kq5xw has branch ENI eni-04adabec0bf4dcf87
STEP: validating pod traffic-server-6948dd5fbb-jzqdh has branch ENI eni-0c22e8d5053121239
STEP: validating pod traffic-server-6948dd5fbb-4gjr9 has branch ENI eni-046672806ebcdfb25
STEP: validating pod traffic-server-6948dd5fbb-6l59b has branch ENI eni-04e57595c226c99a9
STEP: validating pod traffic-server-6948dd5fbb-82qkm has branch ENI eni-01e89686ea648beef
STEP: validating pod traffic-server-6948dd5fbb-x5frk has branch ENI eni-074562574cc5f307c
STEP: validating pod traffic-server-6948dd5fbb-hj855 has branch ENI eni-088dac2b8cb9661ec
STEP: validating pod traffic-server-6948dd5fbb-pvnvn has branch ENI eni-0da8568df438cbf85
STEP: validating pod traffic-server-6948dd5fbb-497rn has branch ENI eni-016c5ab24f538edcf
STEP: validating pod traffic-server-6948dd5fbb-5s7fl has branch ENI eni-0dfca956892d1df29
STEP: validating pod traffic-server-6948dd5fbb-h74xr has branch ENI eni-02f37dea78c59bcc1
STEP: validating pod traffic-server-6948dd5fbb-2xvf5 has branch ENI eni-0b885cceeb79fd2af
STEP: validating pod traffic-server-6948dd5fbb-jcqjn has branch ENI eni-04bd36e0f417c4390
STEP: validating pod traffic-server-6948dd5fbb-rzs7l has branch ENI eni-079b62272d07b6caa
STEP: validating pod traffic-server-6948dd5fbb-wh6rw has branch ENI eni-08bc53143a6be5b18
successfully validated the server pod list
successfully created traffic client
STEP: validating pod traffic-client-regular-pods-9kqgp has branch ENI eni-0612fa2a4eac6ce91
STEP: validating pod traffic-client-regular-pods-hj9dv has branch ENI eni-0755511a83bbd7024
STEP: validating pod traffic-client-regular-pods-hl6gm has branch ENI eni-0c7110f4998b4372a
STEP: validating pod traffic-client-regular-pods-sl85n has branch ENI eni-0f341fc941e7642ef
STEP: validating pod traffic-client-regular-pods-hm2nb has branch ENI eni-0727b35b29d9d7ce6
STEP: validating pod traffic-client-regular-pods-dlpqv has branch ENI eni-08cfbb1d0c4abe0e8
STEP: validating pod traffic-client-regular-pods-w7c7d has branch ENI eni-0f062ea8fb969cfd1
STEP: validating pod traffic-client-regular-pods-67l6z has branch ENI eni-09d9379a2d228b0f3
STEP: validating pod traffic-client-regular-pods-k6x2s has branch ENI eni-08517b0eb5b717b83
STEP: validating pod traffic-client-regular-pods-qtlqp has branch ENI eni-0bebe2a938f9cbb41
STEP: validating pod traffic-client-regular-pods-dslv7 has branch ENI eni-029e6a32b8baed1fc
STEP: validating pod traffic-client-regular-pods-pfcsk has branch ENI eni-03f5c7046fcf5f79a
STEP: validating pod traffic-client-regular-pods-rlvws has branch ENI eni-0c2b116b109a13796
STEP: validating pod traffic-client-regular-pods-8w2rs has branch ENI eni-070d86a821df69dc7
STEP: validating pod traffic-client-regular-pods-4h5x4 has branch ENI eni-0d08f8005f962637d
STEP: validating pod traffic-client-regular-pods-8mbhw has branch ENI eni-09f6b860bbdfd97d8
STEP: validating pod traffic-client-regular-pods-72fxh has branch ENI eni-01ec053528262e5be
STEP: validating pod traffic-client-regular-pods-wflnh has branch ENI eni-0415f4a5683653ab8
STEP: validating pod traffic-client-regular-pods-lznlj has branch ENI eni-0018ee06f8888ce53
STEP: validating pod traffic-client-regular-pods-2bn26 has branch ENI eni-07eab2c5d0bde35c5
STEP: validating pod traffic-client-regular-pods-wb59p has branch ENI eni-0438872246dfd9141
STEP: validating pod traffic-client-regular-pods-z226n has branch ENI eni-09e4390ebf04611fa
STEP: validating pod traffic-client-regular-pods-kz2t2 has branch ENI eni-099c2e2d3773d2cf9
STEP: validating pod traffic-client-regular-pods-d4z4m has branch ENI eni-02930c2596e2ea646
STEP: validating pod traffic-client-regular-pods-g5mvd has branch ENI eni-004b6dd46f266ea03
STEP: validating pod traffic-client-regular-pods-pqgmk has branch ENI eni-002de60fd4ad249f6
STEP: validating pod traffic-client-regular-pods-djq48 has branch ENI eni-02e0d4d3e21ff4d18
successfully validated the server pod list
successfully fetched the metrics from metric server
SuccessRate: 100STEP: deleting test namespace
STEP: waiting for the branch ENI to be cooled down

• [SLOW TEST:115.727 seconds]
Security Group for Pods Test
/Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:34
  when testing traffic between branch ENI and branch ENI pods
  /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:105
    should have 99%+ success rate
    /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:110
------------------------------
Security Group for Pods Test when testing traffic to a port on Branch ENI that's not open 
  should have 0% success rate
  /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:137
STEP: creating test namespace
STEP: creating the Security Group Policy
successfully created server deployment
successfully created metric server pod
successfully fetched list of pod belonging to deployment
STEP: validating pod traffic-server-59988cf69d-68nxr has branch ENI eni-0b9c1b846ec340c96
STEP: validating pod traffic-server-59988cf69d-qw86q has branch ENI eni-0d0c1869f40f85312
STEP: validating pod traffic-server-59988cf69d-2kvzn has branch ENI eni-00af03651353ad999
STEP: validating pod traffic-server-59988cf69d-kmrvz has branch ENI eni-0e72575a8ca2099b4
STEP: validating pod traffic-server-59988cf69d-t9njf has branch ENI eni-0d48280c0d03f1247
successfully validated the server pod list
successfully created traffic client
successfully fetched the metrics from metric server
SuccessRate: 0, Input List: [{0 5 traffic-client-regular-pods-n2rpr [{10.0.192.137:2271 failed to connect to server dial tcp 10.0.192.137:2271: i/o timeout} {10.2.52.100:2271 failed to connect to server dial tcp 10.2.52.100:2271: i/o timeout} {10.2.102.9:2271 failed to connect to server dial tcp 10.2.102.9:2271: i/o timeout} {10.0.214.208:2271 failed to connect to server dial tcp 10.0.214.208:2271: i/o timeout} {10.1.94.239:2271 failed to connect to server dial tcp 10.1.94.239:2271: i/o timeout}]} {0 5 traffic-client-regular-pods-p4knl [{10.0.192.137:2271 failed to connect to server dial tcp 10.0.192.137:2271: i/o timeout} {10.2.52.100:2271 failed to connect to server dial tcp 10.2.52.100:2271: i/o timeout} {10.2.102.9:2271 failed to connect to server dial tcp 10.2.102.9:2271: i/o timeout} {10.0.214.208:2271 failed to connect to server dial tcp 10.0.214.208:2271: i/o timeout} {10.1.94.239:2271 failed to connect to server dial tcp 10.1.94.239:2271: i/o timeout}]}]STEP: deleting test namespace
STEP: waiting for the branch ENI to be cooled down

• [SLOW TEST:95.888 seconds]
Security Group for Pods Test
/Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:34
  when testing traffic to a port on Branch ENI that's not open
  /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:132
    should have 0% success rate
    /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:137
------------------------------
Security Group for Pods Test when toggling DISABLE_TCP_EARLY_DEMUX when disabling DISABLE_TCP_EARLY_DEMUX 
  TCP liveness probe will fail
  /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:237
STEP: creating test namespace
STEP: creating the Security Group Policy
STEP: getting the aws-node daemon set in namesapce kube-system
STEP: setting the environment variables on the ds to map[DISABLE_TCP_EARLY_DEMUX:false]
STEP: updating the daemon set with new environment variable
STEP: creating branch ENI pod with liveliness probe
STEP: validating pod liveliness-pod has branch ENI eni-0606567188452edf1
STEP: waiting for the liveliness probe to succeed/fail
STEP: getting the updated branch ENI pod
STEP: verifying the container restarted 1 times
STEP: getting the aws-node daemon set in namesapce kube-system
STEP: setting the environment variables on the ds to map[DISABLE_TCP_EARLY_DEMUX:false]
STEP: updating the daemon set with new environment variable
STEP: deleting test namespace
STEP: waiting for the branch ENI to be cooled down

• [SLOW TEST:126.181 seconds]
Security Group for Pods Test
/Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:34
  when toggling DISABLE_TCP_EARLY_DEMUX
  /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:158
    when disabling DISABLE_TCP_EARLY_DEMUX
    /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:232
      TCP liveness probe will fail
      /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:237
------------------------------
Security Group for Pods Test when toggling DISABLE_TCP_EARLY_DEMUX when enabling DISABLE_TCP_EARLY_DEMUX 
  TCP liveness probe will succeed
  /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:245
STEP: creating test namespace
STEP: creating the Security Group Policy
STEP: getting the aws-node daemon set in namesapce kube-system
STEP: setting the environment variables on the ds to map[DISABLE_TCP_EARLY_DEMUX:true]
STEP: updating the daemon set with new environment variable
STEP: creating branch ENI pod with liveliness probe
STEP: validating pod liveliness-pod has branch ENI eni-0eae9b8f9b230429c
STEP: waiting for the liveliness probe to succeed/fail
STEP: getting the updated branch ENI pod
STEP: verifying the container restarted 0 times
STEP: getting the aws-node daemon set in namesapce kube-system
STEP: setting the environment variables on the ds to map[DISABLE_TCP_EARLY_DEMUX:false]
STEP: updating the daemon set with new environment variable
STEP: deleting test namespace
STEP: waiting for the branch ENI to be cooled down

• [SLOW TEST:182.436 seconds]
Security Group for Pods Test
/Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:34
  when toggling DISABLE_TCP_EARLY_DEMUX
  /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:158
    when enabling DISABLE_TCP_EARLY_DEMUX
    /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:240
      TCP liveness probe will succeed
      /Users/abhipth/go/src/github.com/aws/amazon-vpc-cni-k8s/test/e2e/pod-eni/securiy_group_per_pod_test.go:245
------------------------------
STEP: disabling pod-eni on aws-node DaemonSet
STEP: getting the aws-node daemon set in namesapce kube-system
STEP: setting the environment variables on the ds to map[ENABLE_POD_ENI:{}]
STEP: updating the daemon set with new environment variable
STEP: deleting the key-pair used to create nodegroup
STEP: deleting the self managed node group
STEP: deleting the security group
STEP: detaching the AmazonEKSVPCResourceController policy from the cluster role

Ran 5 of 5 Specs in 1172.725 seconds
SUCCESS! -- 5 Passed | 0 Failed | 0 Pending | 0 Skipped
PASS

Ginkgo ran 1 suite in 19m38.203258381s
Test Suite Passed

Automation added to e2e:
Yes

Will this break upgrades or downgrades. Has updating a running cluster been tested?:
No

Does this change require updates to the CNI daemonset config files to work?:
No

Does this PR introduce any user-facing change?:
No

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

[jayanthvn]

Sorry a bit late, if not added can we also add a traffic case for node to pod on the same node? This was the tcp v4 early demux issue.