Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: [M3-8098] - Refactor StackScript create test to be resilient to Image deprecations #10788

Merged
merged 4 commits into from
Aug 26, 2024

Conversation

AzureLatte
Copy link
Contributor

@AzureLatte AzureLatte commented Aug 15, 2024

Description 📝

Update StackScript create test to be more resilient to the changes that Images may be deprecated on the backend by using real Image data retrieved directly from the API.

Changes 🔄

  • Remove hardcoded Image choices and use real Image data retrieved directly from the API.
  • Add new util function interceptGetAllImages

How to test 🧪

yarn cy:run -s "cypress/e2e/core/stackscripts/create-stackscripts.spec.ts"

As an Author I have considered 🤔

Check all that apply

  • 👀 Doing a self review
  • ❔ Our contribution guidelines
  • 🤏 Splitting feature into small PRs
  • ➕ Adding a changeset
  • 🧪 Providing/Improving test coverage
  • 🔐 Removing all sensitive information from the code and PR description
  • 🚩 Using a feature flag to protect the release
  • 👣 Providing comprehensive reproduction steps
  • 📑 Providing or updating our documentation
  • 🕛 Scheduling a pair reviewing session
  • 📱 Providing mobile support
  • ♿ Providing accessibility support

@AzureLatte AzureLatte self-assigned this Aug 15, 2024
@AzureLatte AzureLatte requested a review from a team as a code owner August 15, 2024 15:14
@AzureLatte AzureLatte requested review from cliu-akamai and removed request for a team August 15, 2024 15:14
Copy link

github-actions bot commented Aug 15, 2024

Coverage Report:
Base Coverage: 82.62%
Current Coverage: 82.62%

@AzureLatte AzureLatte requested a review from a team as a code owner August 20, 2024 13:32
@AzureLatte AzureLatte requested review from carrillo-erik and jaalah-akamai and removed request for a team August 20, 2024 13:32
@jaalah-akamai jaalah-akamai added the Add'tl Approval Needed Waiting on another approval! label Aug 21, 2024
@mjac0bs mjac0bs self-requested a review August 22, 2024 14:20
Copy link
Contributor

@mjac0bs mjac0bs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not totally certain what's happening here, because this test seems to be failing due to a regions-related issue. I get consistent failures on the first test case locally. If I skip that first test case and only run the second (the only one this PR touches), that did pass. Another case of M3-8471? cc @jdamore-linode

Screenshot 2024-08-22 at 10 39 38 AM

image

When I checked CI, I see that the first and third runners had failing tests. The test failure on the third runner is expected and fixed recently in #10811. I can't see the failures on the first runner due to some CI UI glitch, but by searching for this PR number (10788) in our notifications Slack channel, it looks like this spec is failing there too.

@jdamore-linode
Copy link
Contributor

Another case of M3-8471? cc @jdamore-linode

It sounds like maybe yes and no... when I run the test locally I don't have any issue with Cypress finding the region by its ID, but I do still get failures pretty often but not consistently (seems kinda 50/50). But the fact that it's failing consistently for you makes it sound like you could be experiencing the ID issue -- were you running Cloud with yarn dev/yarn up, or via yarn start:manager:ci? Wondering if you get the same outcome?

In my case, the issue seems to be caused by these two factors:

  • Sometimes the (random) region Cypress tries to select is not visible in the drop-down because of overflow, so we just have to scroll it into view before trying to click on it
  • There seems to be some kind of Cloud issue where the selected region doesn't get honored if it's made really quickly after page load (e.g. as if it's waiting for some API request to resolve). If Cypress selects a region too quickly, the autocomplete drop-down gets dismissed but the region select remains with no value selected. It seems like waiting for the account availability request to resolve before interacting fixes the issue, but that could be incidental

Either way, I'll have a ticket and PR open later this afternoon, and will circle back and review @AzureLatte's improvements to this other test too! We can disregard this failure for the sake of PR review since we are seeing this happen in CI where Azure's changes aren't present.

Copy link
Contributor

@mjac0bs mjac0bs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving since test failures (which are addressed in follow-up PRs) are unrelated to these changes, which look good and pass consistently.

@mjac0bs mjac0bs added Approved Multiple approvals and ready to merge! and removed Add'tl Approval Needed Waiting on another approval! Ready for Review labels Aug 26, 2024
@AzureLatte
Copy link
Contributor Author

Thank you all for the review!

@AzureLatte AzureLatte merged commit 6d76d2c into linode:develop Aug 26, 2024
18 of 19 checks passed
@AzureLatte AzureLatte deleted the M3-8098 branch August 26, 2024 17:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Approved Multiple approvals and ready to merge!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants