Simultaneous creation of multiple vl3-NSEs

[glazychev-art]

Description

In the current algorithm, when creating a new endpoint, a request is sent to the master vl3-endpoint, which is already registered. Master is the first endpoint in the list received from registry. And it can be different for different endpoints. This master allocates CIDR for a new NSE from a global prefix.

So, we have following problems:

1. If we already have 2 endpoints and 2 more new ones appear, we can see CIDR conflict:

2. After solving the first problem, we can see the next one. So we have two concurrent create requests:

If this happens at the same time, neither one nor the other will know about each other, because the request to the registry was at the very beginning.

Possible solution

1. If we will receive list of NSEs from registry in sorted order, we can solve the first problem. All new NSEs will connect to the first of this list.

2. For the second problem, we can use Watch flag in the registry query.
   Steps:

   • Receive the list of registered NSEs
   • Start watching for new ones
   • Do all procedures - connect to the other endpoints, create a server and registration.
   • Stop watching - now it can serve new clients by itself.
   • Check, if there were any new NSEs while watching? If yes, it means that new endpoints were registered before us and we need to connect to all of them.
   • Check, is master the same? If not, we have no guarantee that the new master received requests from new NSEs that originally belonged to the first master. This is necessary for state consistency and CIDR allocation. So, in this case, we need to close all connections and start over.

[glazychev-art]

@edwarnicke @denis-tingaikin @Bolodya1997
Thoughts?

[denis-tingaikin]

I think to solve this we can simply add registration_time for nse that will be set on registry side.

[edwarnicke]

I think to solve this we can simply add registration_time for nse that will be set on registry side.

@denis-tingaikin
two questions:

1. Don't we already have a time in the form of the expiretime?
2. Would the registration_time move on refresh?

[denis-tingaikin]

Don't we already have a time in the form of the expiretime?

No, expiration_time is updating on refresh.

Would the registration_time move on refresh?

no, it wouldnt. registration_time assigns once by the final registry.