[D&D] Save index pattern using proper saved object structure

[abbyhu2000]

Description

Previously the index pattern is only saved as an id in the visualization state. This PR saves the index pattern using proper saved object structure: by saving a indexRefName in the searchSourceJSON in kibanaSavedObjectMeta and also saving the name, type and id of the index pattern as a reference object.

Before:

"wizard" : {
            "title" : "ecommerce_wizard",
            "description" : "",
            "visualizationState" : """{"searchField":"","activeVisualization":{"name":"metric","aggConfigParams":[{"id":"1","enabled":true,"type":"cardinality","params":{"field":"category.keyword"},"schema":"metric"},{"id":"2","enabled":true,"type":"cardinality","params":{"field":"customer_birth_date"},"schema":"metric"},{"id":"3","enabled":true,"type":"terms","params":{"field":"customer_gender","order":"desc","size":5,"otherBucket":false,"otherBucketLabel":"Other","missingBucket":false,"missingBucketLabel":"Missing"},"schema":"group"}]},"indexPattern":"ff959d40-b880-11e8-a6d9-e546fe2bba5f"}""",
            "styleState" : """{"addTooltip":true,"addLegend":false,"type":"metric","metric":{"percentageMode":false,"useRanges":false,"colorSchema":"Green to Red","metricColorMode":"None","colorsRange":[{"from":0,"to":10000}],"labels":{"show":true},"invertColors":false,"style":{"bgFill":"#000","bgColor":false,"labelColor":false,"subText":"","fontSize":60}}}""",
            "version" : 1
          },
          "type" : "wizard",
          "references" : [ ],
          "updated_at" : "2022-08-24T17:47:38.274Z"
        }

Should be:

"wizard" : {
            "title" : "ecommerce_wizard",
            "description" : "",
            "visualizationState" : """{"searchField":"","activeVisualization":{"name":"metric","aggConfigParams":[{"id":"1","enabled":true,"type":"cardinality","params":{"field":"category.keyword"},"schema":"metric"},{"id":"2","enabled":true,"type":"cardinality","params":{"field":"customer_birth_date"},"schema":"metric"},{"id":"3","enabled":true,"type":"terms","params":{"field":"customer_gender","order":"desc","size":5,"otherBucket":false,"otherBucketLabel":"Other","missingBucket":false,"missingBucketLabel":"Missing"},"schema":"group"}]}}""",
            "styleState" : """{"addTooltip":true,"addLegend":false,"type":"metric","metric":{"percentageMode":false,"useRanges":false,"colorSchema":"Green to Red","metricColorMode":"None","colorsRange":[{"from":0,"to":10000}],"labels":{"show":true},"invertColors":false,"style":{"bgFill":"#000","bgColor":false,"labelColor":false,"subText":"","fontSize":60}}}""",
            "version" : 1,
            "kibanaSavedObjectMeta" : {
              "searchSourceJSON" : """{"indexRefName":"kibanaSavedObjectMeta.searchSourceJSON.index"}"""
            },
          },
          "type" : "wizard",
          "references" : [ 
           {
              "name" : "kibanaSavedObjectMeta.searchSourceJSON.index",
              "type" : "index-pattern",
              "id" : "ff959d40-b880-11e8-a6d9-e546fe2bba5f"
            }
           ],
          "updated_at" : "2022-08-24T17:47:38.274Z"
        }

Issues Resolved

resolves #1878

Check List

• New functionality includes testing.
  • All tests pass
    • yarn test:jest
    • yarn test:jest_integration
    • yarn test:ftr
• New functionality has been documented.
• Commits are signed per the DCO using --signoff

[codecov-commenter]

Codecov Report

Merging #2218 (00b1812) into main (b961263) will decrease coverage by 0.00%.
The diff coverage is 58.33%.

@@            Coverage Diff             @@
##             main    #2218      +/-   ##
==========================================
- Coverage   66.78%   66.78%   -0.01%     
==========================================
  Files        3131     3133       +2     
  Lines       60057    60079      +22     
  Branches     9150     9153       +3     
==========================================
+ Hits        40109    40123      +14     
- Misses      17759    17766       +7     
- Partials     2189     2190       +1     

Impacted Files	Coverage Δ
...rd/public/application/utils/get_top_nav_config.tsx	3.03% <0.00%> (-0.31%)	⬇️
...blic/application/utils/use/use_saved_wizard_vis.ts	3.33% <0.00%> (-0.12%)	⬇️
...s/wizard/public/saved_visualizations/_saved_vis.ts	0.00% <ø> (ø)
...d_visualizations/saved_visualization_references.ts	0.00% <0.00%> (ø)
...ns/wizard/server/saved_objects/wizard_migration.ts	87.50% <87.50%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

[ashwin-pc]

Love the PR as a whole. Very surgical. Just a few high level callouts:

1. We should be removign the index pattern from the visualization state
2. we should be writing migrations for existing saved objects that have the older version of the saved object
3. We should have backward compatibility tests/migration tests e.g. https://github.com/opensearch-project/OpenSearch-Dashboards/blob/2.2/src/plugins/visualizations/server/saved_objects/visualization_migrations.ts#L35 and its associated test do something identical to this.

A guide to saved object migrations: https://github.com/opensearch-project/OpenSearch-Dashboards/blob/2.2/src/core/server/saved_objects/migrations/README.md

[joshuarrrr]

This is awesome, a really clean implementation! Did you verify that now the references render correctly in the Saved Object Management UI (feature parity with other visualization types)?

[abbyhu2000]

Removed index pattern from the visualizatonState when we save. Added migration for existing wizard saved object and unit tests.

[ashwin-pc]

Love the changes, nice job Abby! Left a few more comments. Also

1. Can we update the default version of the doc to 2 in wizard/public/saved_visualizations/_saved_vis.ts
2. Have you verified that the migration works with an actual doc with did not have the references? Since this is the first migration we have since we forked, I just want to make sure that migrations still work for us.

[joshuarrrr]

Some general comments, but all the functionality looks great, and a big improvement!

[ashwin-pc]

Ship it! 🚢

[abbyhu2000]

Migration tested: Checked out the main branch and created some wizards that only saves index pattern id in the visualization attribute; checked out the new branch and restart opensearch dashboard without restarting opensearch. The migration will trigger. There are no problems with viewing and editing those old wizards and their saved objects have been altered to the new proper way as well (refname has been saved in kibanaSavedObjectMeta and index pattern object has been pushed to the reference array)

[kavilla]

Awesome!