Device Editor resource caching

[knolleary]

Description

This is a quicker rewrite after GitHub discarded what I wrote on submission of the New Issue form...

We should cache the well-defined static files of Node-RED in the Device editor proxy.

1. When a request is made over the proxy for one of the well-defined files, check the local cache for the given version of Node-RED.
2. If file present, return it straight away
3. If file not present, send request to device as we do today, and cache the result for future requests

This will mean the first device to enabled the editor (for a given Node-RED version) after a restart will pay the (existing) penalty of retrieving the files from the device for the cache, but every subsequent request will hit the cache.

We could, in the future, consider bundling these files in the platform - although that has some complication around how to update the bundled files for new versions of Node-RED.

The approach proposed above should be a pretty quick solution to improve the initial load time of the device editor and also make us more tolerant of slower networks.

Required By

• https://app-eu1.hubspot.com/contacts/26586079/record/0-2/8845845707

[joepavitt]

Reported as a solution to https://flowforgeworkspace.slack.com/archives/C032Q63FGG1/p1731506732697229 where one of our customers are experiencing long first-load times on particular machines via the proxy connection to the Editor

[hardillb]

• Device agent update Send nr version device-agent#327
• Start of Forge app changes Better device proxy cache #4792

Still need to work out how/where to install the versions of @node-red/editor-client. Current code has a hard coded test path to a dir I manually created.

We could populate the Docker containers with a starter cache of versions.

Also this only works with device agent 3.0.x (not 3.0.0 as it depends on FlowFuse/device-agent#327)

My current thought is that we should add create a list of NR versions and pre-populate the Docker containers with them. This will cover the main use case of FFC and FF Dedicated (the 2 places with the most devices).

This will need to be done in 2 places

1. The Dockerfiles in the helm repo
2. The Dockerfiles used for the CI pipeline

(I would like to find a way to make these the same file again, but that is a different problem)

Downsides to this it will increase the size of the Docker containers by about 30mb per Node-RED version.

Current list of NR versions to include I think for the best coverage:

• 3.0.2
• 3.1.14
• 4.0.5
• 4.0.4

I don't think we need to add automatic install of new NR versions for the following 2 reasons

1. We can always add to the list above and push a new container on a new NR release
2. Unless we add a volume mounted on the cache path any installed versions will be lost on next deploy

[joepavitt]

@hardillb can I get a status check please?

[hardillb]

• PoC works (Better device proxy cache #4792)
• depends on version of Device agent that will be released this week.
• Need to decided how/which NR versions to bundle into docker containers (both release containers and CI generated containers)

Basically need feedback on last update

And of course list of NR versions will need fix release planned for this week added.

/cc @knolleary

[hardillb]

Forge app code is up for review
Device agent changes are merged.

Outstanding action is update to helm docker build and CI docker container build.

[hardillb]

Verified on Staging