Performance: Move cluster based socket work into web worker (step 1)

[richard-cox]

Is your feature request related to a problem? Please describe.

• Updates to resources are received to the UI thread via Web Socket
• In various cases there are a LOT of messages coming in.
  • resource.created, resource.change, resource.remove events for resources that change
  • resources that are created/remove cause resource.change to counts type (note BE have fixed an issue where counts updated even if there are no changes)
• Each message, or 1 seconds worth of batched messages, needs to be processed and stored. Once stored watchers and possible screen renders are triggered.
• This all happens in the UI thread which can struggle

Describe the solution you'd like

• Move the handling of the cluster store's socket into a Web Worker
• Optimise how these are sent to the UI thread
  • Batch similar messages
  • Discard changes that are superseded by newer events

Additional context

• This is change number 1 of 4
• Step 2 is Performance: Only update store if a resource type has changed (step 2) #7895
• Step 3 is Performance: Make API requests from within the web worker (step 3) #7896
• Step 4 is Performance: Server-side-like pagination via web worker (step 4) #6541

[richard-cox]

Issues requires update for QA

[git-ival]

Hi @richard-cox and @Sean-McQ, are there any steps or areas to validate for this issue? I'm a bit unclear on how to validate this.

[richard-cox]

@git-ival I think we should try to validate current functionality and ensure there's no regression. The changes revolve around sockets which support how changes to previously fetched resources appear in the UI without the user having to refresh the browser. Some core functionality

• Changes to resources (new, removed, edited) are shown to the users in list and detail views
  • This also covers times when lots (10+ per second) changes happen. They should all be display correctly
  • Resource changes should correctly show in the Cluster Explorer, Cluster Management and API Keys in /dashboard/account
• Disruptions to the connection between browser and Rancher are handled and performance continues after connection is restored
  • For instance after the web socket connections are dropped and re-connected changes to resources are still shown
• The resource count in the cluster explorer's side nav menu is correct following changes to that resource type (added/removed)
• The detail page for Services and Deployments correctly show the Pods list and contain the correct pods

I've tried to avoid too much specific steps, as we may just cover tests we've already done. Generally, anywhere we display a resource should update if it changes somewhere else.

@Sean-McQ Feel free to add any other general area i've missed

[git-ival]

Tested on latest 2.7-head using the 2.7.2-rc1 Rancher chart, LGTM! Created and deleted a few hundred Clusters, Deployments, Pods, Namespaces, and Jobs. Was able to confirm that the resource counts updated in the cluster explorer list views, the sidebar and Cluster Management view