[Embeddables rebuild] Add new registry

[ThomThomson]

Summary

This PR is the next major piece of #167429. It closes #167892 and #167893 by building a new registry for React Embeddables and allowing the Dashboard to render them.

This is the framework that all existing embeddables will be moved onto, and replaces the serialization / deserialization capabilities of the old Embeddable framework. For more information, see the architecture document.

Once this PR merges, the new Embeddable system will be complete.

React Embeddables

Note

The new Embeddable framework is temporarily named React Embeddables to stress their closer coupling with React. It will be renamed to Embeddables once the legacy Embeddables framework is removed.

The React Embeddables framework comes with a number of new systems, which when composed, allows engineers to register serializable components that interact with a container or page. These subsystems are:

The Registry

React Embeddable factories are stored in a registry at the module level of the Embeddable plugin. This means that you can put embeddable in your requiredBundles, import the registration function and call it from anywhere.

import { registerReactEmbeddableFactory } from '@kbn/embeddable';

registerReactEmbeddableFactory('helloWorld', helloWorldFactory);

The Factory

The React Embeddable factory is the replacement for the EmbeddableFactory class defined in src/plugins/embeddable/public/lib/embeddables/embeddable_factory.ts. The new factory is much simpler, taking the form of an object with two methods:

• The deserializeState method takes a package with raw state of an unknown type, a list of references, and a version. This method is meant to replace injectReferences, and migrate and also must cast the final results into the state type of the factory.
• The getComponent async method returns the component that this Embeddable will render. This component should forward an API into its imperative handle.

Parenting

This PR also replaces the parenting system from the legacy Embeddable system. Parenting is now accomplished with a simple React Context. ReactEmbeddableParentContext which must provide a PresentaitonContainer.

When an Embeddable is rendered, specifically during the useReactEmbeddableApiHandle call, it finds its parent via the context and registers itself with the parent. The hook then returns the API it creates, and the parent API.

Instead of parents passing children all of their inherited states, the children get access to the full API of their parent, and from there, they can select, use, or inherit state from their parent as necessary.

Unsaved Changes

This PR also includes a process by which unsaved changes and reset unsaved changes functionality can be automatically built. This takes the form of a useReactEmbeddableUnsavedChanges hook which:

• Requires a Comparator for every piece of state defined in the factory's state type. Comparators are composed of:
  • A publishing subject which provides the current state.
  • A setter function to update this state.
  • Optionally, a custom comparator. If not supplied, the state will be compared with reference equality.
• Gets the last saved state for this uuid from the parent context.
• Sets up subscriptions that compare all state keys every time one of the keys changes, or the last saved state changes.
• Builds a reset function that can set all keys to their last saved state.

An unsaved changes subject is mandatory, but a subject initialized to undefined can be supplied if this Embeddable does not have any state which could change over its lifetime.

Other changes

This PR also makes some changes to the publishingSubject framework and updates all usages.

• The useStateFromPublishingSubject hook is now properly typed so that it returns T | undefined only when either the subject itself, or the subject's contents could be undefined.
• The useBatchedPublishingSubjects hook now uses a tuple so that you can pass any number of publishingSubjects in any order, and they will be batched. This also allows consumers to use arbitrary names for the values which are extracted from the hook.

Hello world code example

interface HelloWorldEmbeddableState {
  name: string
}

const helloWorldFactory: ReactEmbeddableFactory<HelloWorldEmbeddableState> = {
  deserializeState: (statePackage) => statePackage.rawState as HelloWorldEmbeddableState,
  getComponent: (initialState: HelloWorldEmbeddableState, maybeId?: string) => {
    // this is a good place to initialize state.
    const uuid = initializeReactEmbeddableUuid(maybeId);
    const name$ = new BehaviourSubject<string>(initialState.name);

    // this function is async so you could async import in here if necessary
    return RegisterReactEmbeddable((apiRef) => {
      // unwrap state to re-render on change
      const name = useStateFromPublishingSubject(name$);

      // set up unsaved changes. This forces correct types from HelloWorldEmbeddableState
      const { unsavedChanges, resetUnsavedChanges } = useReactEmbeddableUnsavedChanges(
        uuid,
        helloWorldFactory,
        {
          name: [name$, (value) => name$.next(value)],
        }
      );

      // publish API... more in this later
      useReactEmbeddableApiHandle({
        unsavedChanges,
        resetUnsavedChanges,
        serializeState: async () => {
            // this function gets run on save. Serialize name and return it.
            return {
              rawState: {
                name: name$.value,
              }
            };
          }
        }, 
        apiRef,
        uuid
      );

      return <div>HELLO WORLD - I am {name}</div>
    }); 
  }
}

How to test

This PR creates an example React Embeddable in examples/embeddable_examples/public/react_embeddables/eui_markdown_react_embeddable.tsx. The best way to test is to use this React Embeddable, ensuring that all functionality works properly, Unsaved changes, Cloning, Copy to Dashboard, setting titles & description etc.

To get access to this React Embeddable you'll need to yarn start --run-examples. Additionally, there is no registered way to add one to a Dashboard yet. You can import a Dashboard which has one in place to start:

{"attributes":{"description":"","kibanaSavedObjectMeta":{"searchSourceJSON":"{\"query\":{\"query\":\"\",\"language\":\"kuery\"},\"filter\":[]}"},"optionsJSON":"{\"useMargins\":true,\"syncColors\":false,\"syncCursor\":true,\"syncTooltips\":false,\"hidePanelTitles\":false}","panelsJSON":"[{\"type\":\"euiMarkdown\",\"gridData\":{\"x\":0,\"y\":0,\"w\":24,\"h\":15,\"i\":\"839440f3-26ab-48fa-a295-41b70c7a279d\"},\"panelIndex\":\"839440f3-26ab-48fa-a295-41b70c7a279d\",\"embeddableConfig\":{\"content\":\"HELLO_WORLD\"}}]","timeRestore":false,"title":"Test 2","version":1},"coreMigrationVersion":"8.8.0","created_at":"2024-01-12T21:52:10.764Z","id":"904938f1-86a3-40d3-ad4d-dc51084704b7","managed":false,"references":[],"type":"dashboard","typeMigrationVersion":"8.9.0","updated_at":"2024-01-12T21:52:10.764Z","version":"WzgwNCwxXQ=="}
{"excludedObjects":[],"excludedObjectsCount":0,"exportedCount":1,"missingRefCount":0,"missingReferences":[]}

[ThomThomson]

/ci

[elasticmachine]

Pinging @elastic/kibana-presentation (Team:Presentation)

[nreese]

Still working through the PR but wanted to push some of the comments.

[nreese]

How about lazy loading component. Examples should show best practices and best practices recommend keeping bundle size as small as possible.

[ThomThomson]

That's a good point. This markdown example is mostly there for testing purposes at the moment. I've created this follow-up issue to describe further examples that should be built, and an async-imported embeddable is high up on the list.

[nreese]

Why does this return parentApi when parentApi is available at thisApi.parentApi. How about just returning thisApi?

[ThomThomson]

Ah, not sure why I did that. Removed the extra return in 1f94fd4

[nreese]

This should throw if key already exists in registry to prevent plugins from overwriting factories.

[ThomThomson]

Done in 1f94fd4

[nreese]

How about just

return Object.keys(latestChanges).length > 0
  ? latestChanges
  : undefined;

[ThomThomson]

Good call, done in 1f94fd4

[nreese]

Why is the definition stored in an array vs an object? Accessing parts of the definition is cryptic with values like comparators[key]?.[2] instead of accessing a named key like comparators[key]?.comparatorFunction. I would recommend converting this type to an object with named keys unless there is a good reason for an array.

[ThomThomson]

Yes, the cryptic way to access the elements is not ideal.

This is somewhat mitigated by the fact that it's a tuple, not an array, meaning that Typescript can correctly infer the different types of each element.

The reason it's implemented this way is a tradeoff of more complicated definition code for the sake of less verbose adoption code. To illustrate, compare the example below:

Object

const { unsavedChanges, resetUnsavedChanges } = useReactEmbeddableUnsavedChanges(
  uuid,
  markdownEmbeddableFactory,
  {
    firstName: {
      subject: firstName$,
      setter: setFirstName,
    },
    lastName: {
      subject: lastName$,
      setter: setLastName,
    },
    profession: {
      subject: profession$,
      setter: setProfession,
    },
    age: {
      subject: age$,
      setter: setAge,
      comparatorFunction: compareAges,
    },
    otherKey: {
      subject: otherKey$,
      setter: setOtherKey,
    },
  }
);

Tuple

const { unsavedChanges, resetUnsavedChanges } = useReactEmbeddableUnsavedChanges(
  uuid,
  markdownEmbeddableFactory,
  {
    firstName: [firstName$, setFirstName],
    lastName: [lastName$, setLastName],
    profession: [profession$, setProfession],
    age: [age$, setAge, compareAges],
    otherKey: [otherKey$, setOtherKey],
  }
);

Consider that this is meant to be used as a hook inside an already fairly complicated component, that we anticipate some Embeddables to have a lot of state keys, and that the tuple setup is very similar to how other react hooks work.

I've attempted to remove some of the confusion in 1f94fd4 by adding names like const subject = comparator[0]

[nreese]

These changes are really clean and show the benefits of the new embeddable system. I really like how the state diffing is getting pushed down to the embeddable. This should fix a lot of the unsaved changes edge cases.

[nreese]

Should this also check that typeof is function? apiPublishesWritablePanelTitle tests for typeof function. Just wondering if this should be consistent that other interface guards

[ThomThomson]

I haven't been very consistent with these typeof checks to be honest. Mostly so far, I expect the name being present will be enough info for a type guard, but of course there is always the risk that someone makes an API with the exact same key of a different type, which will pass this type guard and end up throwing a runtime error.

In short, I'm not sure how strict to be with these, which is why the inconsistency is there. What do you think?

Either way, I'd expect us to change all of these to be consistent in a followup.

[nreese]

I think its ok as is. We should decide one or the other and then make sure all interfaces are consistent. I would vote to check for function to provide a better guard

[nreese]

How come reactEmbeddableRegistryHasKey is statically imported vs exposed via a plugin contract? Will this have bundle size penalties? Are there state management concerns?

[ThomThomson]

This is a great question!

I'm trying a new strategy here where we keep a registry directly at the module level of a plugin. All plugin code is stateful, so we will end up with just one instance of any code placed in the module.

When exported statically like this, it allows consumers to register their factories anywhere simply by importing and running the function. No prop drilling, fancy contexts, or plugin start contract juggling necessary.

I don't think this has any implications on bundle size, as the code is tiny. As far as I can tell there is the same amount of timing / state management risk using this strategy, as there is using a registry class on the plugin start contract. I.e. a consumer could theoretically register their embeddable factory after a 10000ms delay in both cases.

If we do run into problems down the line, we could fairly painlessly transition to a registry on the plugin contract instead.

[nreese]

This is just a stop gap until this action has been converted? If you had react embeddable then panel notification action would not work until this is converted- right?

[ThomThomson]

Yes exactly, these are just to prevent errors when using any React Embeddable.

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 1f94fd4

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
dashboard	422	424	+2
embeddable	98	105	+7
inputControlVis	66	67	+1
lens	1249	1251	+2
links	118	120	+2
maps	1130	1131	+1
presentationPanel	79	81	+2
visualizations	396	397	+1
total			+18

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id	before	after	diff
@kbn/presentation-containers	33	40	+7
@kbn/presentation-publishing	109	115	+6
embeddable	421	459	+38
total			+51

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
dashboard	383.5KB	383.7KB	+199.0B
lens	1.4MB	1.4MB	+33.0B
presentationPanel	8.1KB	7.8KB	-286.0B
total			-54.0B

Public APIs missing exports

Total count of every type that is part of your API that should be exported but is not. This will cause broken links in the API documentation system. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats exports for more detailed information.

id	before	after	diff
@kbn/presentation-publishing	0	3	+3

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
dashboard	36.9KB	36.9KB	+15.0B
dashboardEnhanced	14.1KB	14.2KB	+85.0B
embeddable	60.7KB	65.4KB	+4.7KB
embeddableEnhanced	7.0KB	7.0KB	+56.0B
presentationPanel	40.5KB	40.4KB	-113.0B
total			+4.8KB

Unknown metric groups

API count

id	before	after	diff
@kbn/presentation-containers	34	42	+8
@kbn/presentation-publishing	143	150	+7
embeddable	521	564	+43
total			+58

async chunk count

id	before	after	diff
dashboard	11	12	+1

ESLint disabled line counts

id	before	after	diff
@kbn/presentation-publishing	2	1	-1
embeddable	4	11	+7
total			+6

Total ESLint disabled count

id	before	after	diff
@kbn/presentation-publishing	2	1	-1
embeddable	6	13	+7
total			+6

History

• 💚 Build #190899 succeeded 98172ae
• 💔 Build #190870 failed 7cff095
• 💔 Build #190846 failed 51794d1
• 💔 Build #190637 failed 0647056
• 💔 Build #190620 failed 991bf4c
• 💔 Build #190619 failed c30d1ef

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[drewdaemon]

LGTM—code review only

[drewdaemon]

If this isn't just temporary, it would be nice to see a comment. If you're going to change this anyway, disregard.

[ThomThomson]

This is just temporary until #175138 is completed. It's there just to avoid a runtime error when a React Embeddable is on the Dashboard.