Moves sockets into the advanced worker (#7760)

* Moves sockets into the advanced worker

* worker can die peacefully now, making switching between cluster work.

* Make waitFor generic, wire in to waitForTestFn

* General Changes
- Fixes for switching cluster
  - includes using common getPerformanceSetting
  - avoid new code to unsub before socket disconnect
- handle `watch` `stop` requests
- lots of TODO's (questions, work, checks, test, etc)
- use common

* Switch socket fixes
- isAdvancedWorker should only be true for cluster store
- advancedWorker to be wired in

* Fix socket id for cluster workers
- sockets use an incremented local var for id
- when we nuke the socket file within the worker this resets, so they all ahve id of 1
- work around this by applying the unix time

* Fix handling of new partical counts response
- seen in dex cluster explorer dashboard
- count cards would be removed when partial counts response received

* Make resourceWatcher the sole location for watch state
- getters canWatch, watchStarted now are worked around (they look at state in the UI thread)
  - we now don't call resource.stop or restart.start in subscription
- tidied up `forgetType`
- moved clearFromQueue from steve mutations into subscription mutations (better location)
- added and removed some TODOs
- fixed watch (stop handler should be higher up, include force watch handling)

* pushes the csrf value into worker and adds it to fetch request headers.

* refactors batchChanges to address ref concerns and be more performant

* Maintain schema reference whilst updating
- This change mutates input in a function, which is bad...
- but ensures the reference isn't broken, which is needed to maintain similar functionality as before

* Fix waitForTestFn
- Seen when creating or viewing clusters

* On unwatch ensure any pending watch requests are removed from the queue
- the probably would have been a problem if the worker wasn't nuked
- however as the codes there lets make it safe

Also added `trace` feature in advanced worker, will probably bring out to other places as well

* Fix navigation from cluster manager world to any cluster
- Ensure that we handle the case where the advanced worker was created but the resource watcher wasn't
- ... but fix case where this was happening (aka ensure that a blank cluster context is ignored)

* Tidy some TODOs

* Add perf settings page
- This will help test normal flow (when advanced worker is disabled)
- Note - setting is now in a bag. This may help us better support further settings (enable client side pagination, etc)
  ```
  advancedWorker: { enabled: false },
  ```

* FIX - Nav from cluster dashboard --> specific event --> cluster dashboard and events not re-subbed
- Ensure we block default handling of resource.start (keep state in resource watcher)

* Tidying up some TODOs

* Adds in a cache and uses it to validate SCHEMA messages before batching.

* Forgot to actually save CSRF to the resourceWatcher when instantiated.

* an empty resource in a batchChange to signal remove

* Move addSchemaIndexFields to and created removeSchemaIndexFields in new file
- this avoids bringing class files into the worker

* Fix disconnect/reconnect
- Remove `syncWatch` (do the watch/unwatch straight away)
- Test/Fix re-sub on reconnect
- Test/Fix growls on disconnect

* Tidying up some TODO's
- including clean of workerQueue on resource.stop (this is SUPER defensive)

* batchChanges will now handle aliases

* Fix pods list - WIP
- ensure podsByNamespace is updated on batchChange

TODO
- the final update to the pod is ignored
- removing a namespace cleans the cache correctly
- disabling advanced worker still works

* Fix pods list - fixes
- ensure podsByNamespace is updated on batchChange

Tested / Fixed
- the final update to the pod is ignored
- removing a namespace cleans the cache correctly
- disabling advanced worker still works

* Tidying TODOs

* Remove default same-origin header
- https://developer.mozilla.org/en-US/docs/Web/API/Request/credentials

* Fixed TODO description

* Refactor subscribe, make it clear which vuex feature relates to what

* Lots of Fixes
- batchChanges fixes
  - fix index is 0 issues (!/!!index)
  - only `set` if we have to
  - ensure we set the correct index after pushing to list
  - ensure map is updated after reducing list size with limit
- podsByNamespace fixes
  - ensure when ew replace... we don't use the same referenced object
- general service resource fixes
  - ensure service's pods list stays up to date with store

* Multiple improvements/fixes
- resourceCache - store the hash instead of the whole object. This means longer load time be reduces memory footprint
- resourceWatcher
  - don't re-sub on socket reconnect if watcher is in error
  - don't sub if watcher is in error
  - don't unwatch for 'failed to find schema' and 'too old' errors
    - this clears the error, we won't to keep it to ensure we don't watch
- Remove #5997 comments, follow on work #7917

* toggle debug, remap alias types, cleaned up comments and console

* Unit tests for batchChanges
Much more scope for some crazy content

* Logging tweaks
- disable logging by default
- initWorker comes in too late to affect initial trace, so just rely on the `debug` to toggle at runtime

Co-authored-by: Richard Cox <richard.cox@suse.com>

shell/assets/translations/en-us.yaml

@@ -6813,6 +6813,10 @@ performance:
     count:
       inputLabel: Resource Threshold
       description: The threshold above which filtering by a namespace is required
+  advancedWorker:
+    label: Websocket Web Worker
+    description: Updates to resources pushed to the UI come via WebSocket and are handled in the UI thread. Enable this option to handle cluster WebSocket updates in a Web Worker in a separate thread. This should help the responsiveness of the UI in systems where resources change often.
+    checkboxLabel: Enable Advanced Websocket Web Worker
 
 banner:
   label: Fixed Banners

shell/config/settings.ts

@@ -141,5 +141,6 @@ export const DEFAULT_PERF_SETTING = {
   forceNsFilter:                {
     enabled:   false,
     threshold: 1500,
-  }
+  },
+  advancedWorker: { enabled: false },
 };

shell/detail/service.vue

@@ -11,7 +11,6 @@ import Tab from '@shell/components/Tabbed/Tab';
 import { CATTLE_PUBLIC_ENDPOINTS } from '@shell/config/labels-annotations';
 import { KEY, VALUE } from '@shell/config/table-headers';
 import { POD } from '@shell/config/types';
-import { allHash } from '@shell/utils/promise';
 import { findBy } from '@shell/utils/array';
 
 export default {
@@ -36,13 +35,7 @@ export default {
   },
 
   async fetch() {
-    const hash = { pods: this.value.pods() };
-
-    const res = await allHash(hash);
-
-    for (const k in res) {
-      this[k] = res[k];
-    }
+    await this.value.fetchPods();
   },
 
   data() {
@@ -159,7 +152,7 @@ export default {
       :weight="4"
     >
       <ResourceTable
-        :rows="pods"
+        :rows="value.pods"
         :headers="podTableHeaders"
         key-field="id"
         :table-actions="false"

shell/models/service.js

@@ -128,23 +128,24 @@ export default class extends SteveModel {
     return out;
   }
 
-  get pods() {
+  get podRelationship() {
     const { metadata:{ relationships = [] } } = this;
 
-    return async() => {
-      const podRelationship = (relationships || []).filter(relationship => relationship.toType === POD)[0];
-      let pods = [];
+    return (relationships || []).filter(relationship => relationship.toType === POD)[0];
+  }
 
-      if (podRelationship) {
-        pods = await this.$dispatch('cluster/findMatching', {
-          type:      POD,
-          selector:  podRelationship.selector,
-          namespace: this.namespace
-        }, { root: true });
-      }
+  async fetchPods() {
+    if (this.podRelationship) {
+      await this.$dispatch('cluster/findMatching', {
+        type:      POD,
+        selector:  this.podRelationship.selector,
+        namespace: this.namespace
+      }, { root: true });
+    }
+  }
 
-      return pods;
-    };
+  get pods() {
+    return this.podRelationship ? this.$getters.matching( POD, this.podRelationship.selector, this.namespace ) : [];
   }
 
   get serviceType() {

shell/pages/c/_cluster/settings/performance.vue

@@ -267,6 +267,22 @@ export default {
             />
           </div>
         </div>
+        <!-- Advanced Websocket Worker -->
+        <div class="mt-40">
+          <h2>{{ t('performance.advancedWorker.label') }}</h2>
+          <p>{{ t('performance.advancedWorker.description') }}</p>
+          <Banner
+            color="error"
+            label-key="performance.experimental"
+          />
+          <Checkbox
+            v-model="value.advancedWorker.enabled"
+            :mode="mode"
+            :label="t('performance.advancedWorker.checkboxLabel')"
+            class="mt-10 mb-20"
+            :primary="true"
+          />
+        </div>
       </div>
     </div>
     <template v-for="err in errors">