[Workspace]Import sample data to current workspace #6105

wanglam · 2024-03-11T09:14:39Z

Description

For now, the import sample data page is a sub page under home application. The home application is not visible inside specific workspace. This PR separate import sample data page and register it as a new standalone application. Then import sample data page can be used inside workspace. This PR added a new entry point for import sample data in workspace overview page. User can visit import sample data page by this link quickly.

Issues Resolved

#6106

Screenshot

The sample data can be imported to specific workspace.

Testing the changes

Since switch workspace by manually wasn't support in current main branch. We need to apply some patch to testing all the changes.

Checkout the branch code and run yarn osd bootstrap
enable workspace and start OSD server

add workspace.enabled: true and savedObjects.permission.enabled: true to the config/opensearch_dashboards.yml to enable workspace feature flag
run yarn start --no-base-path start OSD server to make sure no random base path will be appended

Navigate to workspace creator page and create a test workspace
Click left dropdown switch to created workspace
Click "Discover pre-loaded datasets before adding your own." to visit workspace's Add sample data page

Click all Add data button, all sample data will be imported like below image

There will be four dashboards in current workspace.

Click View data to view imported dashboard like below image
Create another workspace, and visit another workspace's add sample data page like step 4 & 5.

The sample data was not import in the new workspace.

Switch back to the first workspace, click remove button, the sample data will be removed like below image

The [Flights] Global Flight Dashboard will be removed in this workspace like above image.

Changelog

feat: [Workspace]Import sample data to current workspace

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

codecov · 2024-03-11T09:28:39Z

Codecov Report

Attention: Patch coverage is 83.90805% with 14 lines in your changes missing coverage. Please review.

Project coverage is 67.37%. Comparing base (54b5eb5) to head (9a2bd3e).
Report is 432 commits behind head on main.

Files with missing lines	Patch %	Lines
src/plugins/home/public/plugin.ts	16.66%	10 Missing ⚠️
...home/server/services/sample_data/data_sets/util.ts	95.23%	0 Missing and 2 partials ⚠️
.../services/sample_data/data_sets/ecommerce/index.ts	0.00%	1 Missing ⚠️
...erver/services/sample_data/data_sets/logs/index.ts	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6105      +/-   ##
==========================================
- Coverage   67.55%   67.37%   -0.19%     
==========================================
  Files        3428     3443      +15     
  Lines       67344    67762     +418     
  Branches    10996    11024      +28     
==========================================
+ Hits        45497    45653     +156     
- Misses      19177    19439     +262     
  Partials     2670     2670

Flag	Coverage Δ
Linux_1	`33.04% <7.69%> (-0.01%)`	⬇️
Linux_2	`55.13% <ø> (ø)`
Linux_3	`45.15% <7.69%> (-0.01%)`	⬇️
Linux_4	`34.81% <83.90%> (+0.08%)`	⬆️
Windows_1	`33.06% <7.69%> (-0.01%)`	⬇️
Windows_2	`55.10% <ø> (ø)`
Windows_3	`45.16% <7.69%> (-0.02%)`	⬇️
Windows_4	`34.81% <83.90%> (+0.08%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Signed-off-by: Lin Wang <wonglam@amazon.com>

SuZhou-Joe · 2024-03-14T07:52:39Z

src/plugins/home/public/application/components/sample_data_set_cards.js

@@ -81,7 +81,10 @@ export class SampleDataSetCards extends React.Component {
  loadSampleDataSets = async (dataSourceId) => {


It seems we are adding workspaceId into the parameters of listSampleDataSets/install/uninstall but not in loadSampleDataSets, any reason for such difference?

The loadSampleDataSets function call listSampleDataSets below. It will read the current workspace id from the workspace service directly. So we don't add the parameter for workspace id.

SuZhou-Joe · 2024-03-14T07:54:21Z

src/plugins/home/server/services/sample_data/data_sets/ecommerce/index.ts

-    getDataSourceIntegratedSavedObjects: (dataSourceId?: string, dataSourceTitle?: string) =>
-      getSavedObjectsWithDataSource(getSavedObjects(), dataSourceId, dataSourceTitle),


Is it save to delete this field?

This method has been moved to src/plugins/home/server/services/sample_data/data_sets/util.ts, searching all OSD, no other places use it.

src/plugins/home/server/services/sample_data/data_sets/flights/index.ts

src/plugins/home/server/services/sample_data/data_sets/logs/index.ts

SuZhou-Joe · 2024-03-14T07:56:13Z

src/plugins/home/server/services/sample_data/data_sets/util.ts

@@ -5,59 +5,74 @@

 import { SavedObject } from 'opensearch-dashboards/server';

-export const appendDataSourceId = (id: string) => {


Seems this logic has gone?

Yes. This function will be refactor to addPrefixTo in below.

Any updates on this comment?

The appendDataSourceId has been added back. To support workspace, the workspaceId has been added to the appendDataSourceId result function. It will add workspace id prefix at the return value of getDataSourceIntegratedDashboard and getDataSourceIntegratedDefaultIndex functions.

…ta-to-workspace Signed-off-by: Lin Wang <wonglam@amazon.com>

Flyingliuhub · 2024-05-16T15:52:13Z

src/plugins/home/public/application/application.test.tsx

+      </div>
+    `);
+  });
+  it('should call unlisten history after destroy', async () => {


maybe like "should clean up history listeners after unmount" ?

Flyingliuhub · 2024-05-16T16:04:22Z

src/plugins/home/server/services/sample_data/data_sets/util.test.ts

+      getFinalSavedObjects({
+        dataset,
+      })
+    ).toBe(dataset.savedObjects);


should be "toEqual" here too, so can performs a deep equality check?

Use toBe here means we just want to use the original saved objects here. I think toBe would be more strict than toEqual. toBe is for compare object reference, they are the same object if reference is same.

Flyingliuhub · 2024-05-16T16:22:12Z

Is it workspace.enabled: true? maybe you need to update your test steps in above

add workspace.enabled=true to the config/opensearch_dashboards.yml to enable workspace feature flag

Flyingliuhub · 2024-05-16T18:45:05Z

@wanglam do you have a chance to verify the all different cases for import sample data. And how to work with MDS to add sample data too

user1 not link to any workspace, does user1 see the workspace sample indices from users who link to workspace and created sample indices?
user2 create workspace and import sample data, does user2 can see the user1's sample data
user3 create workspace and import sample data, and add user2 to user3's works space, does user2 can see user3's imported sample data?
I tried to assign other users to the created workspace, but cannot find where to add, do you have any introduction for assign user to workspace?

Thanks

Signed-off-by: Lin Wang <wonglam@amazon.com>

wanglam · 2024-05-17T15:34:13Z

@wanglam do you have a chance to verify the all different cases for import sample data. And how to work with MDS to add sample data too

user1 not link to any workspace, does user1 see the workspace sample indices from users who link to workspace and created sample indices?

user2 create workspace and import sample data, does user2 can see the user1's sample data

user3 create workspace and import sample data, and add user2 to user3's works space, does user2 can see user3's imported sample data?
I tried to assign other users to the created workspace, but cannot find where to add, do you have any introduction for assign user to workspace?

Thanks

Sure. I think these three cases would work fine. I will test them. For add user to created workspace, you can follow the test steps of this PR. First of all we need to add savedObjects.permission.enabled: true to the config/opensearch_dashboards.yml file. Then the permission tab will be exists in the workspace create / update form.

For update existing workspace, we can go to the workspace list page. Then click the pencil icon, we will navigate to the workspace update page. Click the Users & Permissions tab in the workspace update page, and then we can add new user to workspace by clicking "Add new" button under Users section like below image:

ruanyl · 2024-05-20T05:19:00Z

src/plugins/home/public/application/application.tsx

+  // dispatch synthetic hash change event to update hash history objects
+  // this is necessary because hash updates triggered by using popState won't trigger this event naturally.
+  // This must be called before the app is mounted to avoid call this after the redirect to default app logic kicks in
+  const unlisten = history.listen((location) => {


Do we need this for import sample data app? It seems it's home page app specific.

Signed-off-by: Lin Wang <wonglam@amazon.com>

wanglam · 2024-05-21T13:13:02Z

@wanglam do you have a chance to verify the all different cases for import sample data. And how to work with MDS to add sample data too

user1 not link to any workspace, does user1 see the workspace sample indices from users who link to workspace and created sample indices?

user2 create workspace and import sample data, does user2 can see the user1's sample data

user3 create workspace and import sample data, and add user2 to user3's works space, does user2 can see user3's imported sample data?
I tried to assign other users to the created workspace, but cannot find where to add, do you have any introduction for assign user to workspace?

Thanks

Hi @Flyingliuhub , I've tested these three scenarios in my local. Here are all the findings:
First of all, for all imported saved objects to specific workspace can be found in Dashboards Management -> Saved Objects list. So for import sample data to workspace, it means these data can be displayed in the dashboards and visualizations list.

For scenario 1, since all imported sample data share the same data index. The user1 can see the imported data indices whatever imported from workspace or out of workspace
For scenario 2, user2 won't see user1 imported sample saved objects includes index patterns, visualizations and dashboards if user1 and user2 not in the same workspace. user2 can't see user1's workspace.
For scenario 3, user3 will see user2 imported sample saved objects after user2 be assigned to the same workspace.

ruanyl · 2024-05-20T05:41:08Z

src/plugins/home/public/application/components/home_app.js

+      <Router>
+        <Switch>
+          <Route
+            path="*"


Q: why does it need a Route with path * here? Seems we don't even need Router here

The renderTutorialDirectory need props.match.params.tab data and pass it to TutorialDirectory component. But all the tabs has been deleted in this PR. I think we can remove the Route here and pass a fixed SAMPLE_DATA_TAB_ID here.

ruanyl · 2024-05-22T14:15:59Z

src/plugins/home/server/services/sample_data/data_sets/util.ts

@@ -5,59 +5,74 @@

 import { SavedObject } from 'opensearch-dashboards/server';

-export const appendDataSourceId = (id: string) => {


Any updates on this comment?

Signed-off-by: Lin Wang <wonglam@amazon.com>

* Import sample data to workspace Signed-off-by: Lin Wang <wonglam@amazon.com> * Enable workspace ui plugin Signed-off-by: Lin Wang <wonglam@amazon.com> * Add changelog for import sample data to current workspace Signed-off-by: Lin Wang <wonglam@amazon.com> * feat: register sample data as standalone app (#8) * feat: register sample data as standalone app Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: optimize code Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: add comment Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: use props to pass homeLink Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: add unit test Signed-off-by: SuZhou-Joe <suzhou@amazon.com> --------- Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * Retrieve workspace id from request Signed-off-by: Lin Wang <wonglam@amazon.com> * Remove workspace id in query Signed-off-by: Lin Wang <wonglam@amazon.com> * Move changelog to fragments Signed-off-by: Lin Wang <wonglam@amazon.com> * Fix sample data list unit tests Signed-off-by: Lin Wang <wonglam@amazon.com> * Remove no need workspaces deps Signed-off-by: Lin Wang <wonglam@amazon.com> * Remove manual created changelogs Signed-off-by: Lin Wang <wonglam@amazon.com> * Changeset file for PR #6105 created/updated * Enable sample data in workspace overview page (#9) * enable sample data in workspace overview page Signed-off-by: Hailong Cui <ihailong@amazon.com> * add comments for empty id Signed-off-by: Hailong Cui <ihailong@amazon.com> --------- Signed-off-by: Hailong Cui <ihailong@amazon.com> * Add unit tests for getFinalSavedObjects in data sets util file Signed-off-by: Lin Wang <wonglam@amazon.com> * Add unit tests for renderImportSampleDataApp destroy Signed-off-by: Lin Wang <wonglam@amazon.com> * Address PR comments Signed-off-by: Lin Wang <wonglam@amazon.com> * Remove history listen in renderImportSampleDataApp Signed-off-by: Lin Wang <wonglam@amazon.com> * Remove Route for workspace import sample data entry point Signed-off-by: Lin Wang <wonglam@amazon.com> --------- Signed-off-by: Lin Wang <wonglam@amazon.com> Signed-off-by: SuZhou-Joe <suzhou@amazon.com> Signed-off-by: Hailong Cui <ihailong@amazon.com> Co-authored-by: SuZhou-Joe <suzhou@amazon.com> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> Co-authored-by: Hailong Cui <ihailong@amazon.com> (cherry picked from commit 3e9a159) Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Import sample data to workspace * Enable workspace ui plugin * Add changelog for import sample data to current workspace * feat: register sample data as standalone app (#8) * feat: register sample data as standalone app * feat: optimize code * feat: add comment * feat: use props to pass homeLink * feat: add unit test --------- * Retrieve workspace id from request * Remove workspace id in query * Move changelog to fragments * Fix sample data list unit tests * Remove no need workspaces deps * Remove manual created changelogs * Changeset file for PR #6105 created/updated * Enable sample data in workspace overview page (#9) * enable sample data in workspace overview page * add comments for empty id --------- * Add unit tests for getFinalSavedObjects in data sets util file * Add unit tests for renderImportSampleDataApp destroy * Address PR comments * Remove history listen in renderImportSampleDataApp * Remove Route for workspace import sample data entry point --------- (cherry picked from commit 3e9a159) Signed-off-by: Lin Wang <wonglam@amazon.com> Signed-off-by: SuZhou-Joe <suzhou@amazon.com> Signed-off-by: Hailong Cui <ihailong@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: SuZhou-Joe <suzhou@amazon.com> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> Co-authored-by: Hailong Cui <ihailong@amazon.com>

BionIT · 2024-05-29T00:36:42Z

src/plugins/home/server/services/sample_data/data_sets/util.ts

-      }
-
-      // update reference
-      if (saveObject.type === 'index-pattern') {


why removing the data source reference for index pattern?

@wanglam Is it expected? If not, can we fix it? #6850

I think that's removed by accident. Will add them back and add some unit tests.

…t#6105) * Import sample data to workspace Signed-off-by: Lin Wang <wonglam@amazon.com> * Enable workspace ui plugin Signed-off-by: Lin Wang <wonglam@amazon.com> * Add changelog for import sample data to current workspace Signed-off-by: Lin Wang <wonglam@amazon.com> * feat: register sample data as standalone app (opensearch-project#8) * feat: register sample data as standalone app Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: optimize code Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: add comment Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: use props to pass homeLink Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * feat: add unit test Signed-off-by: SuZhou-Joe <suzhou@amazon.com> --------- Signed-off-by: SuZhou-Joe <suzhou@amazon.com> * Retrieve workspace id from request Signed-off-by: Lin Wang <wonglam@amazon.com> * Remove workspace id in query Signed-off-by: Lin Wang <wonglam@amazon.com> * Move changelog to fragments Signed-off-by: Lin Wang <wonglam@amazon.com> * Fix sample data list unit tests Signed-off-by: Lin Wang <wonglam@amazon.com> * Remove no need workspaces deps Signed-off-by: Lin Wang <wonglam@amazon.com> * Remove manual created changelogs Signed-off-by: Lin Wang <wonglam@amazon.com> * Changeset file for PR opensearch-project#6105 created/updated * Enable sample data in workspace overview page (opensearch-project#9) * enable sample data in workspace overview page Signed-off-by: Hailong Cui <ihailong@amazon.com> * add comments for empty id Signed-off-by: Hailong Cui <ihailong@amazon.com> --------- Signed-off-by: Hailong Cui <ihailong@amazon.com> * Add unit tests for getFinalSavedObjects in data sets util file Signed-off-by: Lin Wang <wonglam@amazon.com> * Add unit tests for renderImportSampleDataApp destroy Signed-off-by: Lin Wang <wonglam@amazon.com> * Address PR comments Signed-off-by: Lin Wang <wonglam@amazon.com> * Remove history listen in renderImportSampleDataApp Signed-off-by: Lin Wang <wonglam@amazon.com> * Remove Route for workspace import sample data entry point Signed-off-by: Lin Wang <wonglam@amazon.com> --------- Signed-off-by: Lin Wang <wonglam@amazon.com> Signed-off-by: SuZhou-Joe <suzhou@amazon.com> Signed-off-by: Hailong Cui <ihailong@amazon.com> Co-authored-by: SuZhou-Joe <suzhou@amazon.com> Co-authored-by: opensearch-changeset-bot[bot] <154024398+opensearch-changeset-bot[bot]@users.noreply.github.com> Co-authored-by: Hailong Cui <ihailong@amazon.com>

github-actions bot added the first-time-contributor label Mar 11, 2024

wanglam added 2 commits March 11, 2024 18:42

Import sample data to workspace

4321cb4

Signed-off-by: Lin Wang <wonglam@amazon.com>

Enable workspace ui plugin

d29f691

Signed-off-by: Lin Wang <wonglam@amazon.com>

wanglam changed the title ~~[Workspace]Import sample data to workspace~~ [Workspace]Import sample data to current workspace Mar 11, 2024

Add changelog for import sample data to current workspace

54adb59

Signed-off-by: Lin Wang <wonglam@amazon.com>

wanglam force-pushed the feat-import-sample-data-to-workspace branch from dec0efb to 54adb59 Compare March 11, 2024 10:44

wanglam marked this pull request as ready for review March 14, 2024 06:04

wanglam requested review from ananzh, kavilla, AMoo-Miki, ashwin-pc, joshuarrrr, abbyhu2000, zengyan-amazon, zhongnansu, manasvinibs, ZilongX, Flyingliuhub, BSFishy, curq, bandinib-amzn, SuZhou-Joe, ruanyl and BionIT as code owners March 14, 2024 06:04

SuZhou-Joe reviewed Mar 14, 2024

View reviewed changes

wanglam marked this pull request as draft March 14, 2024 08:04

ruanyl added the workspace label Mar 15, 2024

Merge remote-tracking branch 'origin/main' into feat-import-sample-da…

5240160

…ta-to-workspace Signed-off-by: Lin Wang <wonglam@amazon.com>

wanglam marked this pull request as ready for review March 22, 2024 10:21

wanglam requested a review from mengweieric as a code owner May 16, 2024 02:53

Flyingliuhub reviewed May 16, 2024

View reviewed changes

wanglam added 2 commits May 17, 2024 23:12

Address PR comments

28d1f74

Signed-off-by: Lin Wang <wonglam@amazon.com>

Merge branch 'main' into feat-import-sample-data-to-workspace

6162c2d

Merge branch 'main' into feat-import-sample-data-to-workspace

b82fcec

SuZhou-Joe previously approved these changes May 20, 2024

View reviewed changes

ruanyl reviewed May 20, 2024

View reviewed changes

Remove history listen in renderImportSampleDataApp

5f8f9f4

Signed-off-by: Lin Wang <wonglam@amazon.com>

wanglam dismissed SuZhou-Joe’s stale review via 5f8f9f4 May 21, 2024 09:05

Merge branch 'main' into feat-import-sample-data-to-workspace

3f27d71

ruanyl reviewed May 22, 2024

View reviewed changes

wanglam added 2 commits May 22, 2024 23:31

Remove Route for workspace import sample data entry point

4d560c6

Signed-off-by: Lin Wang <wonglam@amazon.com>

Merge branch 'main' into feat-import-sample-data-to-workspace

9a2bd3e

Flyingliuhub approved these changes May 24, 2024

View reviewed changes

ruanyl approved these changes May 24, 2024

View reviewed changes

SuZhou-Joe added backport 2.x v2.15.0 labels May 24, 2024

SuZhou-Joe merged commit 3e9a159 into opensearch-project:main May 24, 2024
71 checks passed

opensearch-trigger-bot bot mentioned this pull request May 24, 2024

[Backport 2.x] [Workspace]Import sample data to current workspace #6826

Merged

BionIT reviewed May 29, 2024

View reviewed changes

wanglam mentioned this pull request Jun 6, 2024

[Workspace] Import sample data to specific workspace #6106

Closed

AMoo-Miki added the in-next label Sep 27, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Workspace]Import sample data to current workspace #6105

[Workspace]Import sample data to current workspace #6105

wanglam commented Mar 11, 2024 •

edited

Loading

codecov bot commented Mar 11, 2024 •

edited

Loading

SuZhou-Joe Mar 14, 2024

wanglam Mar 14, 2024

SuZhou-Joe Mar 14, 2024

wanglam Mar 14, 2024

SuZhou-Joe Mar 14, 2024

wanglam Mar 14, 2024

ruanyl May 22, 2024

wanglam May 22, 2024

Flyingliuhub May 16, 2024

Flyingliuhub May 16, 2024

wanglam May 17, 2024

Flyingliuhub commented May 16, 2024

Flyingliuhub commented May 16, 2024 •

edited

Loading

wanglam commented May 17, 2024 •

edited

Loading

ruanyl May 20, 2024

wanglam commented May 21, 2024 •

edited

Loading

ruanyl May 20, 2024

wanglam May 22, 2024

ruanyl May 22, 2024

BionIT May 29, 2024 •

edited

Loading

BionIT May 29, 2024

wanglam May 29, 2024

		@@ -81,7 +81,10 @@ export class SampleDataSetCards extends React.Component {
		loadSampleDataSets = async (dataSourceId) => {

		getDataSourceIntegratedSavedObjects: (dataSourceId?: string, dataSourceTitle?: string) =>
		getSavedObjectsWithDataSource(getSavedObjects(), dataSourceId, dataSourceTitle),

		@@ -5,59 +5,74 @@

		import { SavedObject } from 'opensearch-dashboards/server';

		export const appendDataSourceId = (id: string) => {

[Workspace]Import sample data to current workspace #6105

[Workspace]Import sample data to current workspace #6105

Conversation

wanglam commented Mar 11, 2024 • edited Loading

Description

Issues Resolved

Screenshot

Testing the changes

Changelog

Check List

codecov bot commented Mar 11, 2024 • edited Loading

Codecov Report

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Flyingliuhub commented May 16, 2024

Flyingliuhub commented May 16, 2024 • edited Loading

wanglam commented May 17, 2024 • edited Loading

Choose a reason for hiding this comment

wanglam commented May 21, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

BionIT May 29, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wanglam commented Mar 11, 2024 •

edited

Loading

codecov bot commented Mar 11, 2024 •

edited

Loading

Flyingliuhub commented May 16, 2024 •

edited

Loading

wanglam commented May 17, 2024 •

edited

Loading

wanglam commented May 21, 2024 •

edited

Loading

BionIT May 29, 2024 •

edited

Loading