feat(ci): add workflow config for auto-updating studios

[cngonzalez]

RESOLVES SDX-1412

Description

As we move to different ways of building the Studio, it's important to test all functionality across existing and new build configuration. This PR adds a matrix option for auto-updating studios (with the potential of adding more). It also adds some conditional scripting logic to ensure that studios build and run according to the matrix configuration parameter.

In addition, it adds and updates tests that depend on a studio built with the auto-updates parameter.

What to review

The changed Github workflow. I have a few specific questions:

1. Are we okay with config: [auto-updating, default]? Are there better names for what we want to express?
2. Should we also add logic for additional datasets depending on whether or not the studio is auto-updating?

Testing

The specific tests added in this PR should pass under both conditions. All other tests should pass under both conditions.

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
page-building-studio	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 26, 2024 6:29pm
performance-studio	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 26, 2024 6:29pm
test-compiled-studio	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 26, 2024 6:29pm
test-next-studio	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 26, 2024 6:29pm
test-studio	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Nov 26, 2024 6:29pm

1 Skipped Deployment

Name	Status	Preview	Comments	Updated (UTC)
studio-workshop	⬜️ Ignored (Inspect)	Visit Preview		Nov 26, 2024 6:29pm

[github-actions]

No changes to documentation

[github-actions]

Component Testing Report Updated Nov 26, 2024 6:30 PM (UTC)

✅ All Tests Passed -- expand for details

File	Status	Duration	Passed	Skipped	Failed
comments/CommentInput.spec.tsx	✅ Passed (Inspect)	48s	15	0	0
formBuilder/ArrayInput.spec.tsx	✅ Passed (Inspect)	9s	3	0	0
formBuilder/inputs/PortableText/Annotations.spec.tsx	✅ Passed (Inspect)	31s	6	0	0
formBuilder/inputs/PortableText/copyPaste/CopyPaste.spec.tsx	✅ Passed (Inspect)	39s	11	7	0
formBuilder/inputs/PortableText/copyPaste/CopyPasteFields.spec.tsx	✅ Passed (Inspect)	0s	0	12	0
formBuilder/inputs/PortableText/Decorators.spec.tsx	✅ Passed (Inspect)	18s	6	0	0
formBuilder/inputs/PortableText/DisableFocusAndUnset.spec.tsx	✅ Passed (Inspect)	10s	3	0	0
formBuilder/inputs/PortableText/DragAndDrop.spec.tsx	✅ Passed (Inspect)	3m 0s	0	0	0
formBuilder/inputs/PortableText/FocusTracking.spec.tsx	✅ Passed (Inspect)	47s	15	0	0
formBuilder/inputs/PortableText/Input.spec.tsx	✅ Passed (Inspect)	1m 44s	21	0	0
formBuilder/inputs/PortableText/ObjectBlock.spec.tsx	✅ Passed (Inspect)	1m 18s	18	0	0
formBuilder/inputs/PortableText/PresenceCursors.spec.tsx	✅ Passed (Inspect)	9s	3	9	0
formBuilder/inputs/PortableText/RangeDecoration.spec.tsx	✅ Passed (Inspect)	26s	9	0	0
formBuilder/inputs/PortableText/Styles.spec.tsx	✅ Passed (Inspect)	18s	6	0	0
formBuilder/inputs/PortableText/Toolbar.spec.tsx	✅ Passed (Inspect)	37s	12	0	0
formBuilder/tree-editing/TreeEditing.spec.tsx	✅ Passed (Inspect)	0s	0	3	0
formBuilder/tree-editing/TreeEditingNestedObjects.spec.tsx	✅ Passed (Inspect)	0s	0	3	0

[github-actions]

⚡️ Editor Performance Report

Updated Tue, 26 Nov 2024 18:33:08 GMT

Benchmark	reference latency of sanity@latest	experiment latency of this branch	Δ (%) latency difference
article (title)	21.7 efps (46ms)	21.1 efps (48ms)	+2ms (+3.3%)	✅
article (body)	56.2 efps (18ms)	53.5 efps (19ms)	+1ms (+5.1%)	✅
article (string inside object)	25.0 efps (40ms)	23.5 efps (43ms)	+3ms (+6.3%)	✅
article (string inside array)	22.7 efps (44ms)	20.8 efps (48ms)	+4ms (+9.1%)	✅
recipe (name)	47.6 efps (21ms)	47.6 efps (21ms)	+0ms (-/-%)	✅
recipe (description)	52.6 efps (19ms)	57.1 efps (18ms)	-2ms (-7.9%)	✅
recipe (instructions)	99.9+ efps (6ms)	99.9+ efps (6ms)	-0ms (-/-%)	✅
synthetic (title)	17.9 efps (56ms)	18.5 efps (54ms)	-2ms (-3.6%)	✅
synthetic (string inside object)	19.2 efps (52ms)	18.5 efps (54ms)	+2ms (+3.8%)	✅

efps — editor "frames per second". The number of updates assumed to be possible within a second.

Derived from input latency. efps = 1000 / input_latency

Detailed information

🏠 Reference result

The performance result of sanity@latest

Benchmark	latency	p75	p90	p99	blocking time	test duration
article (title)	46ms	51ms	64ms	206ms	219ms	12.0s
article (body)	18ms	21ms	26ms	68ms	94ms	5.7s
article (string inside object)	40ms	44ms	50ms	81ms	162ms	7.3s
article (string inside array)	44ms	47ms	65ms	84ms	276ms	7.6s
recipe (name)	21ms	23ms	27ms	41ms	0ms	8.2s
recipe (description)	19ms	20ms	23ms	33ms	0ms	4.7s
recipe (instructions)	6ms	8ms	10ms	11ms	0ms	3.2s
synthetic (title)	56ms	60ms	78ms	396ms	878ms	13.8s
synthetic (string inside object)	52ms	55ms	62ms	490ms	1003ms	8.5s

🧪 Experiment result

The performance result of this branch

Benchmark	latency	p75	p90	p99	blocking time	test duration
article (title)	48ms	51ms	54ms	94ms	65ms	11.3s
article (body)	19ms	21ms	30ms	85ms	102ms	5.5s
article (string inside object)	43ms	46ms	64ms	77ms	206ms	7.6s
article (string inside array)	48ms	52ms	57ms	187ms	141ms	7.7s
recipe (name)	21ms	22ms	26ms	62ms	0ms	7.7s
recipe (description)	18ms	19ms	21ms	31ms	0ms	4.6s
recipe (instructions)	6ms	8ms	9ms	10ms	0ms	3.2s
synthetic (title)	54ms	56ms	63ms	395ms	728ms	13.4s
synthetic (string inside object)	54ms	57ms	87ms	511ms	1189ms	8.7s

📚 Glossary

column definitions

• benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
• latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
• p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
• p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
• p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
• blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
• test duration — how long the test run took to complete.

[cngonzalez]

Closing this one for now since there's some choices to be made about our CI processes etc.