-
Notifications
You must be signed in to change notification settings - Fork 811
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix k8s nat manager logic and add --Xnat-kube-service-namespace
flag
#6088
base: main
Are you sure you want to change the base?
Conversation
Signed-off-by: MAKHONIN Aleksey M <Aleksey.MAKHONIN@raiffeisen.ru>
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
minor suggestions on help messages
besu/src/test/java/org/hyperledger/besu/cli/BesuCommandTest.java
Outdated
Show resolved
Hide resolved
besu/src/test/java/org/hyperledger/besu/cli/BesuCommandTest.java
Outdated
Show resolved
Hide resolved
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com> Signed-off-by: Makhonin Alexey <60808275+alex123012@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com> Signed-off-by: Makhonin Alexey <60808275+alex123012@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com> Signed-off-by: Makhonin Alexey <60808275+alex123012@users.noreply.github.com>
Co-authored-by: Sally MacFarlane <macfarla.github@gmail.com> Signed-off-by: Makhonin Alexey <60808275+alex123012@users.noreply.github.com>
@alex123012 there's a couple of unit tests failing. you should be able to reproduce locally
BesuCommandTest > natManagerServiceNameCannotBeUsedWithNatDockerMethod FAILED BesuCommandTest > natManagerServiceNameCannotBeUsedWithNatNoneMethod FAILED |
code looks ok - @alex123012 how would I verify that it's working? new to kubernetes |
So, you could try to use kind/minikube with https://github.com/Consensys/quorum-kubernetes repo with changed image. I've tested it with ibft2. Something like a plan:
If you encounter any problems, you can write to me in telegram or email (makhonin.a.ru@gmail.com) |
@macfarla Wrote a few notes^ |
@macfarla Hi! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 , but we should make namespace optional in order to be backward compatible with prior behavior.
v1Service -> v1Service.getMetadata().getName().contains(besuServiceNameFilter)) | ||
.findFirst() | ||
.orElseThrow(() -> new NatInitializationException("Service not found")); | ||
api.readNamespacedService(besuServiceName, besuServiceNamespace, null); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think service namespace should be an Optional parameter to the manager. Not having to explicitly specify the namespace is a useful feature IMO, and can prevent breaking existing clustered implementations.
The presence or absence of the namespace can gate the behavior of finding first vs explicitly specify the target namespace
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@alex123012 any word on this?
@cdivitotawela any input on this PR? |
Let me check whether I can run this on minikube and verify, |
Testing successful with Kubernetes in AWS with v24.5.2 of Besu. There was one issue I ran into, I had to name the port 'discovery' or it would throw an error. I would have liked to been able to name it 'discovery-tcp' or 'p2p-tcp'. |
Just wondering after talking to some others about K8S behaviour. Does the namespace need manually specifying via a config option, or can the K8S namespace of the Besu pod just be queried and used in discovering the service endpoint? A K8S service can't serve a pod in a different namespace anyway, so it's not clear if there's a reason to add the new |
PR description
Now the Kubernetes nat manager requires cluster-wide permissions (list service resources in all namespaces). Also, it has a bug: when multiple besu networks are configured in the same cluster with the same service names in different namespaces, the nat manager may set an inappropriate IP address for the besu node due to enumerating services from all namespaces and checking only their names.
I suggest adding a new flag:
--Xnat-kube-service-namespace
to specify a concrete besu node service namespace and not use a list of services, but a get request for a specific service.Fixed Issue(s)
fixes #5002