Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Delete ovs bridge when forwarder stop #279

Closed
wants to merge 1 commit into from

Conversation

ljkiraly
Copy link
Contributor

Description

Cleanup the bridge used to connect NSM endpoints.

Issue link

Related issue: #276

How Has This Been Tested?

  • Added unit testing to cover
  • Tested manually
  • Tested by integration testing
  • Have not tested

Reproduction steps described in the issue

Types of changes

  • Bug fix
  • New functionality
  • Documentation
  • Refactoring
  • CI

Related issue: networkservicemesh#276

Signed-off-by: Laszlo Kiraly <laszlo.kiraly@est.tech>
ljkiraly added a commit to Nordix/cmd-forwarder-ovs that referenced this pull request Nov 10, 2023
Related issue: networkservicemesh/sdk-ovs#276
Related PR: networkservicemesh/sdk-ovs#279

Signed-off-by: Laszlo Kiraly <laszlo.kiraly@est.tech>
ljkiraly added a commit to Nordix/cmd-forwarder-ovs that referenced this pull request Nov 14, 2023
Related issue: networkservicemesh/sdk-ovs#276
Related PR: networkservicemesh/sdk-ovs#279

Signed-off-by: Laszlo Kiraly <laszlo.kiraly@est.tech>
Comment on lines +112 to +117
func DeleteBridge(ctx context.Context, bridgeName string) {
stdout, stderr, err := util.RunOVSVsctl("del-br", bridgeName)
if err != nil {
log.FromContext(ctx).Warnf("Failed to remove bridge %s, stdout: %q, stderr: %q, error: %v", bridgeName, stdout, stderr, err)
}
}
Copy link
Member

@denis-tingaikin denis-tingaikin Nov 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we may convert this function to private and do next:

  1. Create a new chain element or edit one of the existing chain elements if possible.
  2. Add application context to the chain element from step 1. (Note: the application context is https://github.com/networkservicemesh/cmd-forwarder-ovs/blob/main/main.go#L109)
  3. Simply start a new gorouite once that waits for the application context to be done and removes the bridge.

With this change, DeleteBridge will be a private thing, and we will prevent unexpected usage and API leaks that may increase the complexity of the system.

Also, if we edit exist chain element in step1 we could close networkservicemesh/cmd-forwarder-ovs#305

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hello @denis-tingaikin,
In case when OVS is running locally indside forwarder-ovs container it is supervised and started by 'supervisord':
https://github.com/networkservicemesh/cmd-forwarder-ovs/blob/0798082507c276e1904d5cc40bea16c05d71e61e/internal/ovsinit/start.go#L64

I've just tried the following change:

diff --git a/pkg/networkservice/chains/forwarder/server.go b/pkg/networkservice/chains/forwarder/server.go
...
+       go func() { // Cleanup the main nsm bridge
+               <-ctx.Done()
+               exec := kexec.New()
+               if err := util.SetExec(exec); err == nil {
+                       _, stdErr, err := util.RunOVSVsctl("del-br", bridgeName)
+                       if stdErr != "" {
+                               fmt.Printf("Failed to delete interfaces %s", stdErr)
+                               return
+                       }
+                       if err != nil {
+                               fmt.Printf("Failed to run ovsvsctl %v", err)
+                               return
+                       }
+               } else {
+                       fmt.Printf("Failed to init exec %v", err)
+                       return
+               }
+               fmt.Println("deleted nsm-br")
+       }()

Since the context already done, the 'supervisord' and the ovs daemons are stopped, this function will fail on github.com/ovn-org/ovn-kubernetes/go-controller/pkg/util.SetExec. In short: can not execute ovs commands after context was canceled or done. Interesting that seemingly the 'defer' is called before context.Done is sent.
If we decide to implement the proposed way the remainders (nsm-bridge and ovs-system) must be deleted by netlink in case of this (init exec) failure. If the nsm bridge is deleted before ovs stops the 'ovs-system' interface is deleted automatically by ovs.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option should be to delete the bridge interface in cmd-forwarder-ovs main without creating additional API. Updated networkservicemesh/cmd-forwarder-ovs#305.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another option should be to delete the bridge interface in cmd-forwarder-ovs main without creating an additional API. Updated networkservicemesh/cmd-forwarder-ovs#305.

Woot! It sounds smart and easy. I think we may go with networkservicemesh/cmd-forwarder-ovs#305.

@ljkiraly ljkiraly marked this pull request as draft November 28, 2023 09:38
@ljkiraly
Copy link
Contributor Author

networkservicemesh/cmd-forwarder-ovs#305.merged, closing this PR

@ljkiraly ljkiraly closed this Nov 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants