[Meta][2.15] Support Multiple Data Source in OpenSearch Dashboards Plugins

[BionIT]

Context

Since 2.4.0, multiple data source feature is enabled as a core plugin(see RFC #1388 and PR #2334).

With multiple data source enabled, users of OpenSearch Dashboards are able to create data source connections and query local and remote OpenSearch clusters(see documentation https://opensearch.org/docs/latest/dashboards/management/data-sources/). Enabling multiple data source at dashboards plugin would ensure users that have been using the multiple data source feature to have a consistent experience in the dashboard, and allows users to use the same dashboard for different clusters.

This meta issue is used to track the PR/Issues for supporting multiple data source in OpenSearch Dashboards Plugins

• [FEATURE] Support Multiple Data Source in Alerting Dashboards Plugin alerting-dashboards-plugin#879
• [FEATURE] Support Multiple Data Source in Security Analytics Dashboards Plugin security-analytics-dashboards-plugin#885
• [FEATURE] Support Multiple Data Source in Feature Anywhere anomaly-detection-dashboards-plugin#731
• [FEATURE] Support Multiple Data Source in Feature Anywhere alerting-dashboards-plugin#965
• [FEATURE] Support Multiple Data Source in Metrics Analytics Dashboards Plugin dashboards-observability#1754

Data source connection picker

In #5717, we introduced a picker to select data connection. This picker is within the data source plugin bundle, and when loaded, it will fetch data source connection which is a type of saved object and then populate the available data connections as single selected options in the drop down.

In order to consume this picker from a plugin, follow the steps as below:

1. in the opensearch_dashboards.json file, add dataSource as required bundle, eg. "requiredBundles": ["dataSourceManagement"]
2. import the component in the file where this picker is needed, eg. import { ClusterSelector } from '../../data_source_management/public'
3. this picker exposes onSelectedDataSource which can be used to get the selected option from the picker component

Data source client

Data source plugin was released in OpenSearch Dashboards 2.4.0 as part of the multi data source feature.
The plugin exposes OpenSearch client wrapper and this is the design https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/docs/multi-datasource/client_management_design.md, it can be acquired from context via the following:
client: OpenSearchClient = await context.dataSource.opensearch.getClient(dataSourceId);
For legacy client, we can initiate the client via the following
apiCaller: LegacyAPICaller = context.dataSource.opensearch.legacy.getClient(dataSourceId).callAPI;
Note that we need to pass in the data source id as argument, and this id can be acquired by passing a function to the child component by following instructions above.

PoC

PoC commit for query bench dashboards plugin to support MDS: opensearch-project/dashboards-query-workbench@main...BionIT:dashboards-query-workbench:mdinteg

A few things to notice:

1. to consume the client, we need to add "optionalPlugins": ["dataSource"] in opensearch_dashboards.json in the plugin repo
2. to consume the React component to pick data source, we need to add "requiredBundles": ["dataSourceManagement"]
3. during plugin set up, in order to register the plugin API schema/spec, we can do the following dataSource.registerCustomApiSchema(sqlPlugin)
4. most plugins might need to depend on the data source picker, please follow to set it up, and once we acquire the selected data source id, we need to pass it to plugin const {dataSourceId} = request.query; then the datasource Id can be passed to the data source client wrapper context.dataSource.opensearch.getClient(dataSourceId)

[hdhalter]

Looks like this is also related to: opensearch-project/documentation-website#7301

[BionIT]

Looks like this is also related to: opensearch-project/documentation-website#7301

Yes, will put together a PR for the documentation update by end of tomorrow the latest

[BionIT]

This is the PR opensearch-project/documentation-website#7328 @hdhalter