[Observability Onboarding] Migrate e2e Playwright tests from oblt-playwright repo #203616

mykolaharmash · 2024-12-10T15:31:22Z

Closes #199016

This change migrates and and expands tests from oblt-playwright repo.

These tests are part of the Nightly workflow and being run by an Ensemble story on the CI.

The Nightly workflow itself is still in development and does not support some of the use cases, that's why kubernetes tests are skipped for now in this PR.

See the ./README.md on how to run the tests locally.

github-actions · 2024-12-10T15:31:37Z

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

/oblt-deploy : Deploy a Kibana instance using the Observability test environments.
run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

elasticmachine · 2024-12-10T15:31:58Z

Pinging @elastic/obs-ux-logs-team (Team:obs-ux-logs)

flash1293 · 2024-12-11T08:39:25Z

Hey, thanks for this! I tested both auto-detect and k8s flows and the while the auto-detect one was working fine, the kubernetes one opened the table flyout for the nodes visualization to early and it was still empty. Refreshing the page and opening the flyout manually made the test pass, I guess we need a similar retry loop here like the one you added for auto-detect and the host overview page?

Other than that LGTM

mykolaharmash · 2024-12-11T14:18:34Z

Hey, thanks for this! I tested both auto-detect and k8s flows and the while the auto-detect one was working fine, the kubernetes one opened the table flyout for the nodes visualization to early and it was still empty. Refreshing the page and opening the flyout manually made the test pass, I guess we need a similar retry loop here like the one you added for auto-detect and the host overview page?

Other than that LGTM

@flash1293 thank you for testing! I've added a retry logic to the k8s test, please take a look

flash1293

LGTM, didn't test again but the change looks like what I had in mind

pheyos

packages/kbn-test/src/functional_test_runner/lib/config/run_check_ftr_configs_cli.ts changes LGTM

wayneseymour

Left some questions.

wayneseymour · 2024-12-13T16:34:44Z

.../plugins/observability_solution/observability_onboarding/e2e/playwright/playwright.config.ts

+  testDir: './',
+  outputDir: './.playwright',
+  /* Run tests in files in parallel */
+  fullyParallel: true,


I see you'd like it to be fully parallel, which I believe is at the worker scope.
Is it ok that below workers is set to 1?

I guess I'm thinking how can it be fully parallel with one worker.

Good point! Tbh, I did not write this config in the first place (migrated from the oblt-playwright repo), but I see this is the recommended basic config from the Playwright docs. I think it makes sense though in our use case, because tests are run both on the CI and locally, so there will be a single worker for the CI (AFAIK the instance running it does not have a lot of resources) and locally it will be the default value, which probably calculated depending on CPU cores count.

wayneseymour · 2024-12-13T16:35:30Z

.../plugins/observability_solution/observability_onboarding/e2e/playwright/playwright.config.ts

+    testIdAttribute: 'data-test-subj',
+    video: {
+      mode: 'off',
+      size: { width: 1920, height: 1200 },


Do we need a size if this config is turned off?

True, we don't need it, will remove the whole video block as it's off by default anyway

wayneseymour · 2024-12-13T16:36:55Z

x-pack/plugins/observability_solution/observability_onboarding/e2e/playwright/stateful/auth.ts

+  if (isAuthenticated) {
+    await page.context().storageState({ path: STORAGE_STATE });
+  } else if (spaceSelector) {
+    await page.locator('xpath=//a[contains(text(),"Default")]').click();


Why use an xpath locator if you've already added data-test-subj as the test id for the framework?

wayneseymour · 2024-12-13T16:39:12Z

...tion/observability_onboarding/e2e/playwright/stateful/pom/components/header_bar.component.ts

+  }
+
+  public readonly helpMenuButton = () =>
+    this.page.locator('xpath=//div[@data-test-subj="helpMenuButton"]');


Basically the same question as before, why use xpath? I could be wrong but I recall it being rather slow in the past and we've got a ton of issues with resources in testing.

wayneseymour · 2024-12-13T16:46:24Z

..._solution/observability_onboarding/e2e/playwright/stateful/pom/pages/onboarding_home.page.ts

+
+  private readonly useCaseKubernetes = () =>
+    this.page.locator(
+      'xpath=//div[@data-test-subj="observabilityOnboardingUseCaseCard-kubernetes"]//input[@type="radio"]'


You know, I just realized that our Team's POC is the only place I know of in the repo that has a playwright-data-test-subj integration.

Here it is.

In this case the kbn/scout package does not override playwrights test id attr, but this test subj functionality is exposed.

Maybe this is useful for you all?

\cc @dmlemeshko

Those are good points about using xpath, I left them as they were originally written in the oblt-playwright repo, but it 100% makes sense to just use test id helper and narrow down using Locator's methods, I'll re-write those selector

…aywright Update to the latest tests Ignore playwright test results Fix output directory Temporarily move playwright config to the root of the repo Enable screenshots Add Kubernetes EA test Reduce timeout for debugging

mykolaharmash · 2024-12-18T13:23:13Z

/oblt-deploy

elasticmachine · 2024-12-18T15:37:41Z

💚 Build Succeeded

Buildkite Build
Commit: ce7c493
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-203616-ce7c493e913b

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`observabilityOnboarding`	273.8KB	274.0KB	+217.0B

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`@kbn/observability-onboarding-e2e`	3	4	+1

Total ESLint disabled count

id	before	after	diff
`@kbn/observability-onboarding-e2e`	3	4	+1

History

💔 Build #261469 failed 5c3dee5
💛 Build #260226 was flaky 1ad6ae2
💛 Build #259873 was flaky b7f2c14
💚 Build #259373 succeeded 1e1e4be
💔 Build #259344 failed 341c705

…ywright repo (elastic#203616) Closes elastic#199016 This change migrates and and expands tests from [oblt-playwright](https://github.com/elastic/oblt-playwright) repo. These tests are part of the [Nightly workflow](https://github.com/elastic/ensemble/actions/workflows/nightly.yml) and being run by an Ensemble story on the CI. The Nightly workflow itself is still in development and does not support some of the use cases, that's why kubernetes tests are skipped for now in this PR. See the `./README.md` on how to run the tests locally. --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

mykolaharmash requested a review from a team as a code owner December 10, 2024 15:31

mykolaharmash requested a review from a team December 10, 2024 15:31

botelastic bot added the ci:project-deploy-observability Create an Observability project label Dec 10, 2024

mykolaharmash added backport:skip This commit does not require backporting Team:obs-ux-logs Observability Logs User Experience Team labels Dec 10, 2024

mykolaharmash added the release_note:skip Skip the PR/issue when compiling release notes label Dec 10, 2024

mykolaharmash force-pushed the 199016-moving-tests-auto-detect-only branch from 2a97c3c to 3c5ab27 Compare December 11, 2024 14:17

mykolaharmash force-pushed the 199016-moving-tests-auto-detect-only branch 2 times, most recently from 341c705 to ea353e6 Compare December 11, 2024 15:09

mykolaharmash requested a review from a team as a code owner December 11, 2024 15:09

flash1293 approved these changes Dec 12, 2024

View reviewed changes

mykolaharmash force-pushed the 199016-moving-tests-auto-detect-only branch 2 times, most recently from b7f2c14 to dec550d Compare December 13, 2024 09:26

mykolaharmash requested a review from a team as a code owner December 13, 2024 09:26

mykolaharmash force-pushed the 199016-moving-tests-auto-detect-only branch from dec550d to de469e9 Compare December 13, 2024 09:37

mykolaharmash removed the request for review from a team December 13, 2024 13:42

pheyos approved these changes Dec 13, 2024

View reviewed changes

wayneseymour requested changes Dec 13, 2024

View reviewed changes

wayneseymour approved these changes Dec 17, 2024

View reviewed changes

mykolaharmash added 6 commits December 17, 2024 16:44

Add page reload to the k8s test

e3e186a

Add the Playwright config to IGNORED_PATHS

e586830

Unskip k8s test

1ffe5ce

Remove the default .env file and update README

5582444

Rewrite locator selectors without using xpath

43fbac1

mykolaharmash force-pushed the 199016-moving-tests-auto-detect-only branch from 1ad6ae2 to 43fbac1 Compare December 17, 2024 15:44

[CI] Auto-commit changed files from 'node scripts/yarn_deduplicate'

5c3dee5

Merge branch 'main' into 199016-moving-tests-auto-detect-only

ce7c493

mykolaharmash enabled auto-merge (squash) December 18, 2024 14:28

mykolaharmash merged commit 4bb6521 into elastic:main Dec 18, 2024
8 checks passed

kibanamachine added the v9.0.0 label Dec 18, 2024

mykolaharmash mentioned this pull request Dec 18, 2024

[Observability Onboarding] Expand e2e tests to check data on the dashboard #199808

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Observability Onboarding] Migrate e2e Playwright tests from oblt-playwright repo #203616

[Observability Onboarding] Migrate e2e Playwright tests from oblt-playwright repo #203616

mykolaharmash commented Dec 10, 2024 •

edited

Loading

github-actions bot commented Dec 10, 2024

elasticmachine commented Dec 10, 2024

flash1293 commented Dec 11, 2024 •

edited

Loading

mykolaharmash commented Dec 11, 2024

flash1293 left a comment

pheyos left a comment

wayneseymour left a comment

wayneseymour Dec 13, 2024

mykolaharmash Dec 17, 2024

wayneseymour Dec 13, 2024

mykolaharmash Dec 17, 2024 •

edited

Loading

wayneseymour Dec 13, 2024

wayneseymour Dec 13, 2024

wayneseymour Dec 13, 2024

mykolaharmash Dec 17, 2024

mykolaharmash commented Dec 18, 2024

elasticmachine commented Dec 18, 2024 •

edited

Loading

ESLint disabled line counts

Total ESLint disabled count

[Observability Onboarding] Migrate e2e Playwright tests from oblt-playwright repo #203616

[Observability Onboarding] Migrate e2e Playwright tests from oblt-playwright repo #203616

Conversation

mykolaharmash commented Dec 10, 2024 • edited Loading

github-actions bot commented Dec 10, 2024

🤖 GitHub comments

elasticmachine commented Dec 10, 2024

flash1293 commented Dec 11, 2024 • edited Loading

mykolaharmash commented Dec 11, 2024

flash1293 left a comment

Choose a reason for hiding this comment

pheyos left a comment

Choose a reason for hiding this comment

wayneseymour left a comment

Choose a reason for hiding this comment

wayneseymour Dec 13, 2024

Choose a reason for hiding this comment

mykolaharmash Dec 17, 2024

Choose a reason for hiding this comment

wayneseymour Dec 13, 2024

Choose a reason for hiding this comment

mykolaharmash Dec 17, 2024 • edited Loading

Choose a reason for hiding this comment

wayneseymour Dec 13, 2024

Choose a reason for hiding this comment

wayneseymour Dec 13, 2024

Choose a reason for hiding this comment

wayneseymour Dec 13, 2024

Choose a reason for hiding this comment

mykolaharmash Dec 17, 2024

Choose a reason for hiding this comment

mykolaharmash commented Dec 18, 2024

elasticmachine commented Dec 18, 2024 • edited Loading

💚 Build Succeeded

Metrics [docs]

Async chunks

ESLint disabled line counts

Total ESLint disabled count

History

mykolaharmash commented Dec 10, 2024 •

edited

Loading

flash1293 commented Dec 11, 2024 •

edited

Loading

mykolaharmash Dec 17, 2024 •

edited

Loading

elasticmachine commented Dec 18, 2024 •

edited

Loading