Set up test jail doc & skip current flaky tests #6774
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
Preview this change https://demo.audius.co/jd/test-jail |
DejayJD
commented
Nov 22, 2023
|
|
||
| ## Current Inmates | ||
|
|
||
| #### [Upload Track](./uploadTrack.cy.ts) |
There was a problem hiding this comment.
Should we make a client team ticket to fix the upload track suite?
DejayJD
commented
Nov 22, 2023
| @@ -65,7 +65,8 @@ const completeUpload = () => { | |||
| }).should('exist') | |||
There was a problem hiding this comment.
I noticed that only one upload test was running due to a it.only. I'm assuming that was intentional? But I guess it might have been an accidental push.
Either way that test was still flaky so I skipped the whole suite for now
DejayJD
changed the title
|
Preview this change https://demo.audius.co/jd/test-jail |
|
|
DejayJD
requested review from
a team and
amendelsohn
and removed request for
a team and
amendelsohn
dylanjeffers
approved these changes
Nov 27, 2023
|
Preview this change https://demo.audius.co/jd/test-jail |
rickyrombo
reviewed
Nov 27, 2023
| ## Test Jail Process | ||
|
|
||
| - Create an appropriate ticket for addressing the test | ||
| - Update this doc with the flaky tests and a description |
There was a problem hiding this comment.
I don't think this is necessary if we have the ticket tbh, or if we do want a description, the comment could have it
rickyrombo
reviewed
Nov 27, 2023
dharit-tan
approved these changes
Nov 29, 2023
|
Preview this change https://demo.audius.co/jd/test-jail |
audius-infra
pushed a commit
that referenced
this pull request
Dec 2, 2023
[97ee16b] Fix routing for legacy sign-on (#6834) Dylan Jeffers [c5f150f] [C-3374] Fix play/pause on trending mobile (#6833) Dylan Jeffers [3612090] [C-3417] Cover photos default to blurred profile photo if unset (#6811) Andrew Mendelsohn [fb74614] [PROTO-1458] Expose postgres upgrade logs in mediorum (#6830) Theo Ilie [6deaeab] [C-2591] Fix Follow All button on mobile (#6831) Andrew Mendelsohn [d952d16] [C-3453] Replace edit collection form with new fields (#6829) Andrew Mendelsohn [3ed1747] Make mobile buy buttons stick to bottom (#6827) Reed [4d02605] [PAY-2209] Use single modal for purchase and transfer (#6810) Reed [f84f24e] [PAY-2227] Set min-width for NavPopupMenu (#6826) Reed [17ca613] Auto-hide scheduled releases and index as utc (#6824) Isaac Solo [3d8fbaa] [C-3421] Add email-in-use hint (#6825) Dylan Jeffers [b88999f] [PAY-2215] Play full purchased tracks from mobile library (#6821) Reed [46c953e] Bump version to 0.5.29 audius-infra [1d688a2] Track number of blobs served from mediorum and expose metric endpoint (#6816) Michelle Brier [958ff2b] Fix merchant id (#6822) Raymond Jacobson [d168c29] [C-3403] Fix sign-up e2e (#6815) Dylan Jeffers [534a405] [PAY-2228] Add coinflow merchant id (#6820) Raymond Jacobson [fe57135] Fix analytics after vite migration (#6819) Raymond Jacobson [16fc48c] [C-3399] Fix trending links and twitter embed (#6818) Dylan Jeffers [6517400] [C-3414] Update password reset modals to use harmony components (#6817) Kyle Shanks [7f37ee2] Fix embed player playlist styles (#6813) Raymond Jacobson [1e66d4f] Bump version to 0.5.28 audius-infra [f519ae2] [PAY-2213] Fix closing Add Funds modal (#6814) Raymond Jacobson [28d7f35] Set up test jail doc & skip current flaky tests (#6774) JD Francis [0ca4675] Make DN a thin container when running locally, add createSender on DN start (#6335) Marcus Pasell [8df08b3] trending: use aggregate tables (#6806) Steve Perkins [16e2d08] [C-3389] Native harmony layout components (#6808) JD Francis [e863af4] [C-3419] Add common sign-up components (#6809) Dylan Jeffers [53772e6] [C-3415] Add confirm handle page (#6797) Dylan Jeffers [3e82b69] Negative error codes for allowing (#6807) Isaac Solo [1b7c76f] add arm builds for uptime (#6805) alecsavvy [eadeadb] Update payment router readme (#6804) Saliou Diallo [fc28641] [PAY-2208] Make payment router test e2e (#6803) Raymond Jacobson [19eed3f] Bump version to 0.5.27 audius-infra [d786864] [C-3391] Update native harmony Text component to use harmony theme values (#6798) Kyle Shanks [40c3e18] [PAY-2211] Remove borders around withdraw/purchase tab empty states (#6796) Reed [30a6674] Improve harmony font consistency (#6802) Raymond Jacobson [8c377a4] Protocol dashboard entrypoint (#6776) Michelle Brier [81ae2a1] [PAY-2219] Address Embed QA (#6801) Raymond Jacobson [a5e5d56] Fix FixedDecimal.toShorthand() for numbers less than a tenth (#6742) Marcus Pasell [00b0d25] Add Solana Relay to Discovery (#6782) Marcus Pasell [f2a8150] Fix redirect protocol (#6800) Raymond Jacobson [9ddbf19] [PAY-2220] Use slot_diff in user_bank health (#6799) Reed [12f39f1] fix: Update welcome modal cover photo to use saved cover photos (#6781) JD Francis [0c70996] [PAY-2216] DN health_check 500's if index user bank unhealthy (#6795) Reed [7f35a50] Bump version to 0.5.26 audius-infra [d8cf15c] require push-uptime to deploy (#6794) Michelle Brier [76de38b] [C-3354, C-3396, C-3397] Sign up QA (#6780) Dylan Jeffers [051c6c2] [PAY-2212] Don't grow fullWidth buttons (#6791) Reed [5fe1c83] Fix uptime ts errors (#6793) Michelle Brier [1124ae5] Fix dashboard prod deploy (#6787) Michelle Brier [2fefdad] Enable embed ci workflow on main (#6792) Raymond Jacobson [9b00279] [PROTO-1449] Make mediorum query eth directly for service providers (#6788) Theo Ilie [5bc0a23] [PAY-2176] Implement remote config hooks via context (#6778) Randy Schott [d734058] Default retry for verified user bot (#6783) Isaac Solo [d58dbb4] [PAY-2210] Change crpyto transfer modal button to secondary (#6790) Reed [9624aaa] Fix user bank indexing (#6789) Raymond Jacobson [8448c37] [PAY-2214] Change require to import in AnimatedIconbutton (#6786) Reed [26f89a8] Fix web lint (#6784) Reed [22a34b1] [PAY-2066] Buy Crypto via SOL recovery flow (#6598) Marcus Pasell [b47402a] Connect social to finish profile page, fix account header z index C-3384 (#6779) nicoback2 [d4d6b71] Bump version to 0.5.25 audius-infra [c846f7a] d contracts interface (#6767) Michelle Brier
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
A short doc outlining a short process for maintaining a "test jail"; aka flaky tests that we're skipping until we address their flakiness
Additionally, I skipped the currently flaky tests and added them to the current "inmate" list
Some additional thoughts I had
Do we want to keep the skipped tests in place where they are?
Right now I left the skipped tests wherever they were before, but we could also move the questionable test runs into a separate folder to make it a bit easier to find them. Or just the bottom of the file as well maybe.
My only fear with moving the files around is that it may mean they don't get touched in any refactoring and it could make it more work to reintroduce back into the suite.
That being said, I could see it both ways; having clean test files with 0 skipped code and a separate folder with all the skipped stuff also seems nice for organizational purposes.