feat: IATR cloud data backend #25316

warrensplayer · 2022-12-30T17:56:21Z

Closes [IATR](M0) Fetch run data with failed tests from Cloud #24440

User facing changelog

N/A merging to feature branch

Additional details

Background
This PR moves the logic for determining the "relevant" run for the new Debug page to the server and exposing it through a GraphQL query. The server handles fetching the git hashes to send to the Cypress cloud and processing the runs returned to determine the appropriate run(s) for the front end.

What is a "relevant" run?
A "relevant" run for the Debug page is determined by looking up runs in a Cypress Cloud project that have a Git commit sha that matches one of the last 100 commit shas in the users's current development project. If the current environment does not use Git, then the Debug feature will not be available for that project. See the RelevantRunsDataSource to see the logic for how the "current" and "next" relevant runs are determined.

Main Components

packages/data-context/src/sources/RelevantRunsDatasource.ts
- Server data source that determines which CloudRuns match the local Git shas and should be used for the Debug page
packages/data-context/src/sources/RelevantRunSoecsDatasource.ts
- Server data source that pulls spec counts for the current and next runs. Used by a poller that backs a new GQL subscription
packages/app/src/composables/useRelevantRun.ts
- GQL query for querying for the relevant run order to use the runNumber in subsequent queries
packages/app/src/composables/useTestsForDebug.ts
- GQL query for finding the failed tests to use for a filter in the runner when filtering in debug mode

GQL Updates/Additions

CurrentProject
- added testsForReviewBySpec
- TODO: Need to complete resolver for testsForReviewBySpec. Currently just a placeholder
Mutation
- added movetoNextRelevantRun for front end to indicate that the cached run just be removed and the newest run used as the current run
RelevantRun
- New field type added for representing the relevant current and next run
Subscription
- added relevantRuns to allow for the front end to see changes to the current and next relevant runs
- added relevantRunSpecChange to allow for the banner and page that are showing updating spec counts for RUNNING builds to update

Dependency Updates

Added mocha-junit-reporter and moch-multi-reporters to the data-context package directly
- This was required to fix a bug when a test would fail an assertion. These reporters were being referenced from the root package.json. They would make a call out to the peer dependency for mocha which is currently an older version at the root of the project. Various versions of mocha are used throughout the mono-repo that should be looked at.

Other Updates
Updated the various parts of the Debug feature that was using hardcoded runNumbers to use the new useRelevantRuns composable and delay the initial queries until the value is found.

Steps to test

In order to test this PR, you will currently need to have the Cypress Cloud running locally or point to Cloud staging

Open Cypress and log in to your local Cloud instance
Open a project that has recorded runs
Verify that the Debug page and the sidebar badge represent the appropriate "relevant" run based on the algorithm defined above and documented in the code in RelevantRunsDataSource
Start recording a new run to this project
Verify that the new RUNNING run is shown in the banner on the Debug page as running (banner implemented here [IATR](M1.0) Running/completed build banner for debug page #24853)
When the run completes, verify that the banner updates to show the completed status and a "View run" link (banner implemented here [IATR](M1.0) Running/completed build banner for debug page #24853)
Click the "View run" and verify that the latest run is now shown as the current run on the Debug page

How has the user experience changed?

N/A Backend changes only

PR Tasks

Have tests been added/updated?
Has the original issue (or this PR, if no issue exists) been tagged with a release in ZenHub? (user-facing changes only)
Has a PR for user-facing changes been opened in cypress-documentation?
Have API changes been updated in the type definitions?

# Conflicts: # packages/app/src/debug/DebugContainer.vue # packages/app/src/pages/Debug.vue

…ixed in

…rning test names for review

cypress · 2023-01-04T22:49:51Z

Test summary

26535 • 2 • 1180 • 0 • 46

Run details


Project	cypress
Status	Failed
Commit	`a97a39d`
Started	Jan 17, 2023 6:55 PM
Ended	Jan 17, 2023 7:14 PM
Duration	19:27 💡
OS	Linux Debian -
Browser	Multiple

View run in Cypress Dashboard ➡️

Failures

Run group: launchpad-e2e (Linux, Chrome )
	cypresse2e/migration.cy.ts	1
1	global mode > migrates 2 projects in global mode
Run group: app-ct (Linux, Chrome )
	srcdebug/GroupedDebugFailedTest.cy.tsx	1
1	> mounts correctly and shows artifacts on hover

Flakiness

	create-from-component.cy.ts	2
1	... > runs generated spec
2	... > runs generated spec
	runner/reporter-ct-vite.errors.cy.ts	1
1	Vite - errors ui > cy.readFile
	e2e/origin/commands/navigation.cy.ts	1
1	cy.origin navigation > #consoleProps > .go()
	commands/net_stubbing.cy.ts	1
1	network stubbing > intercepting request > can delay and throttle a StaticResponse
This comment includes only the first 5 flaky tests. See all 46 flaky tests in the Cypress Dashboard.

This comment has been generated by cypress-bot as a result of this project's GitHub integration settings. You can manage this integration in this project's settings in the Cypress Dashboard

mike-plummer

Is working great locally minus the issues I pointed out on Slack. Just a couple comments and it looks like there's a couple from Zach still pending as well, will approve assuming they are answered

packages/graphql/src/schemaTypes/objectTypes/gql-Mutation.ts

packages/graphql/src/schemaTypes/objectTypes/gql-Subscription.ts

packages/graphql/src/schemaTypes/objectTypes/gql-CurrentProject.ts

packages/data-context/src/sources/RelevantRunSpecsDataSource.ts

marktnoonan · 2023-01-17T16:24:41Z

I caught a brief flash of "new" here before this resolved to 0. Not blocking but possibly we could have a little delay to make sure nothing is coming from the cloud before we show the "new" status.

warrensplayer · 2023-01-17T16:49:17Z

I caught a brief flash of "new" here before this resolved to 0. Not blocking but possibly we could have a little delay to make sure nothing is coming from the cloud before we show the "new" status.

@marktnoonan Yes, there are some scenarios causing "flashes" of content. I am going to file a follow up issue for that.

marktnoonan · 2023-01-17T17:02:11Z

What is the expectation when more than one run is pending? We just see the newest in this page?

packages/app/src/debug/DebugFailedTest.vue

marktnoonan

Tested as described in the PR and everything works. I have a few questions and captured them on the fly here: https://www.loom.com/share/bd58758e23d8469d9300dbe5fb37288d - it's a long one but might be good for reference.

~~There are a few instances where state flickers and I think some strategic debounces or waits could help us out as we polish.~~ Nvm, I saw our earlier convo that we'll make a new issue.

The main usability issue I see at the the moment is the need to manually click a button called "View Run" (which is different to the other button "View in Cypress Cloud") on the just-finished run in order to "promote it" to be the run that you are actively debugging.

It feels weird that when Cypress "knows about a run" in the Debug page, it's possible for the run's page not to know about that run yet, and that there's no way to click through to the Cloud from the pending run. I think this has to do with the fact that we don't poll in the background when the user is not on the runs page, and we don't immediately poll again when they come back to it.

None of these are blocking issues for this PR, just a head start on feedback on the feature overall I guess.

warrensplayer · 2023-01-17T18:33:01Z

Merging this to the feature branch. Items left to complete are filed here:

#25486
#25487

warrensplayer and others added 17 commits December 21, 2022 09:39

Basic relevant run field in place

d16fe12

Merge branch 'feature/IATR-M0' into stokes/IATR-M0-cloud-run-data

2318faa

# Conflicts: # packages/app/src/debug/DebugContainer.vue # packages/app/src/pages/Debug.vue

Merge branch 'feature/IATR-M0' into stokes/IATR-M0-cloud-run-data

6035b86

Introduce useRelevantRun composable

03be489

Remove test yarn script

9c0123a

Remove other test yarn script

b91fd55

Merge from feature branch and updates for new form of Relevant Runs m…

3e9f2c1

…ixed in

Merge branch 'feature/IATR-M0' into stokes/IATR-M0-cloud-run-data

30f5e4d

Renaming fields for relevantRuns field and adding composable for retu…

48a563e

…rning test names for review

Update IATR release date

0e23ecb

Revert projectId

b9bcb11

Updating subscription to use same logic as query

1b8ef17

Merge branch 'feature/IATR-M0' into stokes/IATR-M0-cloud-run-data

05540c4

Adding support for "next" run

26fc728

Adding GitDataSource tests and fixing subscription

3835679

Add descriptions to GQL object

ff16fe1

Adding support for saving the current run

fd84e72

warrensplayer added 7 commits January 4, 2023 21:45

Support scenario of only finding a RUNNING CloudRun

e98530f

Fixing type error

f071945

Fixing tests

255532c

Do not try to fetch git log if git has errored

e669b3e

Updating relevant runs logic

1f9566b

Adding comments

476aeae

Additional run logic and comment

33f6a89

warrensplayer marked this pull request as ready for review January 5, 2023 14:12

warrensplayer requested review from a team, mike-plummer, astone123 and marktnoonan January 5, 2023 14:12

warrensplayer added 6 commits January 13, 2023 09:47

Fix polling for new run, test and type fixes

8d6f5d6

Debug statements and removing unused gql field

291a82a

Fix for badge flickering through states on load

23e16ce

Fixing scenario for initial running build

ea29436

Handle being offline plus fixing release date

e9f2cb6

Minor changes for useTestsForDebug

c7700c0

warrensplayer requested a review from rockindahizzy January 13, 2023 20:02

mike-plummer approved these changes Jan 13, 2023

View reviewed changes

warrensplayer added 10 commits January 16, 2023 16:47

Adding padding below debug specs list

bd5c9ac

Updating text for testing type in spec header

5639934

Fixing to use clearTimeout

142da67

PR comments

09c311e

Reloading git hashes after branch change

2fb26a3

Removing unused code and updating debug statements

61f9f82

Fixing error handling

94a9bc4

Fix type check

3611b8e

Fixing test

d6461e5

Fix data-context tests

8786a12

marktnoonan reviewed Jan 17, 2023

View reviewed changes

packages/app/src/debug/DebugFailedTest.vue Show resolved Hide resolved

marktnoonan approved these changes Jan 17, 2023

View reviewed changes

Merge branch 'feature/IATR-M0' into stokes/IATR-M0-cloud-run-data

1ba7d7b

warrensplayer added 2 commits January 17, 2023 13:40

Make sure empty state shows when isHidden is true

a694fca

Removing whitespace

a97a39d

warrensplayer merged commit 1c9d06f into feature/IATR-M0 Jan 17, 2023

warrensplayer deleted the stokes/IATR-M0-cloud-run-data branch January 17, 2023 19:02

warrensplayer mentioned this pull request Jan 18, 2023

feat: wiring up test filters to the Debug page back end #25492

Merged

1 task

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: IATR cloud data backend #25316

feat: IATR cloud data backend #25316

warrensplayer commented Dec 30, 2022 •

edited

Loading

cypress bot commented Jan 4, 2023 •

edited

Loading

mike-plummer left a comment

marktnoonan commented Jan 17, 2023

warrensplayer commented Jan 17, 2023

marktnoonan commented Jan 17, 2023

marktnoonan left a comment •

edited

Loading

warrensplayer commented Jan 17, 2023

feat: IATR cloud data backend #25316

feat: IATR cloud data backend #25316

Conversation

warrensplayer commented Dec 30, 2022 • edited Loading

User facing changelog

Additional details

Steps to test

How has the user experience changed?

PR Tasks

cypress bot commented Jan 4, 2023 • edited Loading

Test summary

Run details

Failures

Flakiness

mike-plummer left a comment

Choose a reason for hiding this comment

marktnoonan commented Jan 17, 2023

warrensplayer commented Jan 17, 2023

marktnoonan commented Jan 17, 2023

marktnoonan left a comment • edited Loading

Choose a reason for hiding this comment

warrensplayer commented Jan 17, 2023

warrensplayer commented Dec 30, 2022 •

edited

Loading

cypress bot commented Jan 4, 2023 •

edited

Loading

marktnoonan left a comment •

edited

Loading