[MD]Multi data source integration with Dev tool

[kristenTian]

No description provided.

[zhongnansu]

Task breakdown

• POC: prove dev_tool can be refactored to replace Node.js http client with opensearch-js client, to interact with OpenSearch, while not having compatibility or stability issue Refactor dev tool console to use opensearch-js client to send request #3544

• Replace http client with opensearch-js client in dev tool console #3546

• Server side impl to integrate with multiple datasource

• UI/UX, need to build a good user experience for user to select a datasource to send query against. Given the cli nature of dev tool console, one idea is we can make changes to the request syntax to allow user to pass datasource info, as either a url path or a query parameter (I realized it's not a good idea because even tho the UX change looks straightforward, but it actually drastic. The allowed query url syntax in dev tool is the same as user directly querying OpenSearch. Through right-click, "copy as curl" button, user should be able to run the same query against OpenSearch. But with datasource added, it's not possible, there's no such param/concept called "datassource" in OS side. Making this change will cause confusion)

  We could consider adding a dropdown list on dev tool page for user to select datasource of his/her current session.

  

Research Notes

• After reading through the console plugin(dev tool) codebase. I realized console plugin doesn't interact with OpenSearch via opensearch-js client. It's just making http requests using Node.js http client. This brings concerns that to onboard dev tool console with multiple datasource, we first need to refactor original logic to replace http client with opensearch-js client, and then extend it to use the data source client on demand. But this may introduce compatibility or stability issues.

Notes:

• Make generic request using client.transport.request

var client = new opensearch.Client();
client.transport.request({ 
  path: '/_search_with_clusters',
  method: 'PUT'
}).then(console.log.bind(console));

cc: @zengyan-amazon

[zhongnansu]

@KrooshalUX fyi for some updates above

[KrooshalUX]

Here are some thoughts expressed in wires (not yet approved, will be reviewing with @kgcreative)

Note: I did consider adding "Data source" to "Settings" but didn't think it created the most obvious/at a glance change for the user.

Option 1 - Most likely a long term solution
Here is what I believe to be the most correct update for Dev Tools, but given the release timeline I am not sure this is practical. Inspired by some IDE designs, this solution introduces OuiControlBar as the "home" for functions like "Show/Hide history", "Settings" (gear) and "Help" (iInCircle). Settings would remain within OuiModal and Help would remain OuiFlyout (layered over the OuiControlBar.) It also corrects the page title from a tab to a proper H1/Title.

In the example of control bar found here and using references for how sources are changed in other features within OpenSearch Dashboards, it appears the best case for changing the data source would be to use an OuiButton (Empty) to reveal OuiPopover containing OuiSelectable with the searchable prop and singleSelection props set to true.

Show history reveals the near-identical UI as present today (minus the close button) within the control bar (the control bar slides up from bottom, as demo'd here (click "Toggle example"). I am not sure if the space to the left of the history is there on purpose - if not, the history would span the width of the window including proper padding.

Option 2 - Most practical for next release timing
In the short term I imagine we are only able to execute something like this using OuiComboBox with the compressed & prepend props set to true. The prepend prop allows for a string value, in this case "data source" or "index pattern" (if its limited to working with index patterns as many of our features are) or something similar might work well to show what source the Dev Tool is interacting with.

@zhongnansu Please let me know your thoughts on these approaches - I can provide a more detailed hand-off once we choose a direction.

[KrooshalUX]

Additional thoughts after a quick review with @kgcreative - is to add tabs under console so users don't need to repeatedly switch between sources, rather create a new tab that uses a different source and the user can switch between.

Notes on this experience : We don't currently have a configurable tab component or pattern. The current OuiTabs is currently for content organization and isn't interactive. So, there is added engineering scope here as well.

Here is what it might look like once we move ahead with the Consolidated Dev Tools project:

[zhongnansu]

@KrooshalUX Thanks Kroosh for the design. Option 2 with datasource dropdown list looks good to me, and the implementation seems straightforward as well. In the additional notes, you mentioned adding tabs to dev tool console, and OuiTabs seems insufficient and it's not interactive. I need to evaluate the effort for implementing such tab component.

cc: @zengyan-amazon @kgcreative @seraphjiang

[zhongnansu]

@KrooshalUX given the time and resources, we are not able to complete the "added engineering scope" of adding tabs. Are you OK with we stick with the original scope for 2.7, to only add data source selector on the top right? Team can work on tabs in 2.8 release.
cc: @kgcreative

[KrooshalUX]

@zhongnansu Yes, that aligns to our expectation. To clarify, that was just ideation, our preference is to move forward with Option 2 from the first comment.

There is another project that may address the other design options, so no need to include this in your roadmap.