[skip-ci] POC: Using expression types to "migrate" state passed in args.

[lukeelmers]

POC: Do not merge!

This POC demonstrates one possible way that pieces of state provided as
args to an expression function could be automatically "migrated" from a
legacy shape without the author of the function needing to know about
it.

The sample combineFilters is useless but serves as a simple example:
it takes an arbitrary number of stringified filter objects, and merges them
together, returning a single filter object.

The argument type is set to filter, which still works because it leverages
the interpreter's built-in casting capabilities, which allows us to cast
from a string to a parsed Filter object.

While we are doing that, we can also examine the shape of the provided
object and migrate it if needed.

You can test this PR by running Kibana, and in Canvas put in a debug
element, then modify the expression as follows:

combineFilters
  "{\"version\": 2, \"value\": \"foo\"}"
  "{\"version\": 3, \"query\": \"bar\"}"
| render as=debug

The filter labeled "version 2" will be automatically migrated to
"version 3" before it is passed to the combineFilters function body.

For illustrative purposes, I've included the original filter args inside
the and: [] array in the returned filter, so you can see how they looked as
they were passed into the function:

[lukeelmers]

The key here is that we still specify types: ['filter'] (not string).

The filter type in the types registry will handle parsing & migrating it automatically for us.

If for some reason this function needed a legacy filter shape, either multiple types could be registered as a means of tracking version (types: ['filter_v1']), or the author could just migrate it backwards in the function body itself (maybe the "down" migration function would also be provided by the owner of Filters). Still not sure what the best solution is for this case.

[lukeelmers]

This is some throwaway code to run a fake migration which simply increments the version number. If version 1 is provided, we consider this to be no longer supported and throw an error. If version 2 is provided, we provide the ability to migrate it up to the latest (version 3).

How these migrations are structured and standardized is a separate topic; this is only an example.

[lukeelmers]

The whole point here is that the "owner" of filters can set up the casting/migrations without the author of the combineFilters function needing to know anything: they just receive the correct, up-to-date filter shape as an object inside their function body as long as they specify their arg as a filter type.

[lukeelmers]

Also worth pointing out that @streamich has floated the idea of making type converters pluggable by third parties in some future version of the expressions service:

registerTypeConverter('string', 'filter', (string) => { /*...*/ })

[stacey-gammon]

I think this could be built on top of the general migration system, but my point was, who is writing migrate: (legacyFilter: any) => ({ ...legacyFilter, version: 3 }), function and what is the implementation?

I think it would work if it used the generic persistableStateFactory we talked about. Code could look something like

 from: {
    string: s => {
    try {
      const { type, ...state } = JSON.parse(s);
      const persistableStateFactory = persistableStateFactories.get(type);

      if (!persistableStateFactory) {
        throw new Error('NOT SUPPORTED');
      }

      if (persistableStateFactory.isDeprecated()) {
        console.warn('LEGACY - upgrade soon to avoid breakage!');
      }
      
      let migratedState = state;
      while (persistableStateFactory.isDeprecated()) {
        const { type: newType, state: migratedState } = persistableStateFactory.migrate(state);
        persistableStateFactory = persistableStateFactories.get(newType);
      }
       return migratedState;
    } catch (e) {
      throw new Error('Unable to parse filter string.');
    }
  }  
}

I was thinking persistableStateFactory.create(state) would return an instance of whatever the factory was for, but since canvas doesn't deal with instances, in this case you'd probably just return the migrated state.

[elasticmachine]

Pinging @elastic/kibana-app-arch (Team:AppArch)

[lukeelmers]

One other caveat I will add is that generally, I don't think we should be leaning on stringified objects being passed to functions. While we can handle these magically with casting, I think in most cases it is clearer to either split the necessary items out into separate args, or to use a subexpression to generate the desired state where possible.

For example, something like this would be more in line with the direction we've been moving with expressions thus far:

combineFilters
  {makeFilter version=2 value="foo"}
  {makeFilter version=3 query="bar"}
| render as=debug

Where makeFilter is a subexpression that handles assembling the state object into the correct shape, and presumably migrating as well. This is similar to the strategy we've taken with the visDimension function.

[lukeelmers]

Closing as we are working out our approach for migrations & reference injection/extraction in #72438