⚠️ [PAY-2421] Upload stems in parallel #7644

rickyrombo · 2024-02-17T04:57:14Z

Description

Moves the uploading of stems to happen in parallel to uploading the parent tracks.

The linear ticket requests that the association of stems wait until the end of the upload, but I took a different approach:

Generate IDs ahead of time for each track
All tracks and stems go through the same handleUploads call, so both are included in the progress reports.
At the end of uploading, check for failed tracks and delete any stems of them

This has its downsides:

We will upload stems even if the parent track fails, and with a parent track ID that does not exist but may exist in the future ⚠️ very bad
- Mitigated with deleting the stems on failure, but of course in critical errors we might not get to the clean up code
- Still leaves the file on storage but maybe that gets pruned? Unsure! But this is a problem from before regardless
It's hackily using _generateId despite that being a "private" method, meaning if someone comes along and changes that method thinking it's safe (since they changed the local calls) then we'll break upload without any type errors ⚠️ very bad
- Mitigated with a stern comment!! Please do not ignore. Maybe we should make this public?
The id is generated earlier in the process, so maybe there's more chance that someone else tries to use that ID at the same time, making a race condition which causes indexing the new track to fail (this actually doesn't seem to have really changed much actually, less worried about this).
We can't add a foreign key constraint to the stems table ⚠️ very bad if we do this
- need to remember this so we don't introduce this bug!!! ⚠️ ⚠️

If you make me, I can do what collections did and wait until all the tracks are uploaded before writing the stems metadata... but maybe that should be a follow on.

Along the way fixed the following bugs:

We were only initializing the upload progress when the file begun uploading. Since we cap workers, this meant the progress could move backwards as we init the upload progress for more files later
We were initializing the artwork filesize to the track filesize (this is masked since we aren't weighting the artwork uploads at all)
We were rounding up on the progress bar, which meant we lingered a bit longer at 100%
Multiple track uploads would take you to a profile that wouldn't be refreshed with your tracks and new track count
Errors on multiple track uploads meant they would never finish
Errors on all tracks on multiple track uploads were reported as successes in analytics
Actions/reducers incorrectly typed stems as TrackMetadata[] instead of StemUploadWithFile[][]
Upload failures weren't setting the error state in the reducer

Made the following enhancements:

Everything goes through handleUpload now, no more single track upload difference
Toast instead of "Something went wrong" screen when all uploads fail
- Can change this back but inadvertently removed the error screen when removing single track upload
Prefetch the uploaded tracks

Things I would do if I could spend more time on it:

Migrate saga to Typescript
Remove calls to AudiusBackend and call libs uploader directly (or maybe even SDK)
Audit the analytics
Maybe take a look at the collections stuff?
Write some tests?

How Has This Been Tested?

Tested both multiple and single track uploads w/ and w/o stems in web against stage and verified stems on the individual track page. Also forced an error for tracks named 'fail' to test error handling cases. I could add some secret title that simulates failures if we want to test that going forward...

Use same handleUpload call to upload both tracks and stems, and remove stems of failed tracks

Stops the finish page from spinning forever by letting it treat "error" similarly to "complete", and having the saga mark the track as errored if it fails

raymondjacobson

nice PR! so many fixes.

I am not worried about the ID collision stuff, but I do think we should make that method public

raymondjacobson · 2024-02-21T01:17:44Z

packages/libs/src/api/Track.ts

@@ -583,6 +585,7 @@ export class Track extends Base {
    this.creatorNode.validateTrackSchema(metadata)
  }

+  // WARNING: CLIENT USES DIRECTLY IN upload/sagas.js !!!


let's make this public.
I also wonder why we want to call it generateTrackId instead of getUnclaimedId?

in retrospect, looks like I could call libs.discoveryProvider.getUnclaimedId('tracks') directly? Should I just do that

lol actually this method is only "private" by convention, the _. I removed that and just made it completely public, feels fine 🤷 Can revisit when we remove this entirely and move to SDK I think.

raymondjacobson · 2024-02-21T01:20:30Z

packages/web/src/common/store/upload/sagas.js

+    const parentTrackId = allUploads[i]?.metadata?.stem_of?.parent_track_id
+    const category = allUploads[i]?.metadata?.stem_of?.category
+    if (!trackIds.includes(parentTrackId)) {
+      console.debug(


should this be console error so we get details in sentry?

maybe.. though maybe the error should be from the place that failed the upload? This is if the stem uploaded successfully and was deleted because the parent was gone

kk yeah nvm

raymondjacobson · 2024-02-21T01:21:07Z

packages/web/src/common/store/upload/sagas.js

+  if (trackIds.length === 0 || trackIds.every((t) => t === undefined)) {
+    yield put(uploadActions.uploadTrackFailed())
+    yield put(
+      toastActions.toast({ content: 'Something went wrong.', type: 'error' })


what happens if this toasts? do we just stay on the progress bar page? that seems not great b/c the user may be on another tab and miss it? I think the error page makes more sense here

no strong feelings here, just felt it's easier to try again w/ clicking upload than taken super far out of the app experience, and figured we were trying to move away from that page and towards toasts. also wasn't sure of mobile behavior (still need to test that)

can you click upload when you get into this though? i think it would show a spinner

yeah it's always there on the sidebar - the action above fails the upload (not sure what happens on mobile though tbh, testing)

I think if you're already on the upload page though and it fails, it wouldn't be obvious to click that again would it?

Not qualified to answer that haha, but changed to put handleError and redirect regardless

I think the ideal solution here would be to have some similar hero to the success one rather than a redirect but that's beyond the scope here

Yes 100% agreed. I think we could do a lot more to map upload errors. There are some that are also fatal and will fail on retry, e.g. 429

audius-infra · 2024-02-21T18:43:08Z

Preview this change https://demo.audius.co/mjp-stems-parallel

gitguardian · 2024-02-21T22:20:54Z

⚠️ GitGuardian has uncovered 2 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
9349019	Triggered	Generic High Entropy Secret	`b455f4d`	packages/ddex/.env.stage	View secret
9412812	Triggered	Generic Password	`b455f4d`	packages/discovery-provider/ddl/local-test.sh	View secret

🛠 Guidelines to remediate hardcoded secrets

Understand the implications of revoking this secret by investigating where it is used in your code.
Replace and store your secrets safely. Learn here the best practices.
Revoke and rotate these secrets.
If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

following these best practices for managing and storing secrets including API keys and other credentials
install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Our GitHub checks need improvements? Share your feedbacks!}

audius-infra · 2024-02-21T22:38:49Z

Preview this change https://demo.audius.co/mjp-stems-parallel

[3ff422f] [PROTO-1513] Setup e2e test stub with LocalStack, wire it up to CI, and automate Mongo init (#7705) Theo Ilie [e705bcf] Allow empty adc auto-deploy commits (#7709) Michelle Brier [ebf162f] [PAY-2523] CollectionTrackRelations migration for sellable albums (#7677) Reed [7fcc5f9] Fix harmony system-appearance (#7706) Dylan Jeffers [f8df062] Update welcome drawer close functionality to always open the notifications drawer (#7704) Kyle Shanks [1845dfc] Remove flipper from android and update readme (#7703) Sebastian Klingler [ced4135] Add documents path to asset url check (#7702) Randy Schott [c3e46e6] Bump version to 0.6.45 audius-infra [81383fd] [C-3879] Remove web Text component (#7686) Dylan Jeffers [138f7c8] Fix half-load of spring animation on all <Page> components (#7692) Raymond Jacobson [cf8ad1b] Fix album notification entity type (#7580) Isaac Solo [4f2a415] [Web] Fix red caret on Transactions History page (#7693) nicoback2 [50156e5] [PAY-2435] Remove pointer cursor when dropzone at max capacity (#7695) Saliou Diallo [ea3365a] [C-3838] Analytics events for mobile sign up (#7655) JD Francis [40edef7] [C-3877] Remove web icon component (#7674) Dylan Jeffers [58c7e47] [C-3880] Add bottom padding to the safe area on the sign in screen (#7689) Kyle Shanks [3e7f71e] Fix app site association (#7694) Sebastian Klingler [4f92b88] [C-3855] Fix android white screen issue on sign up (#7683) JD Francis [596f9cc] Add remote redux debugging to mobile app (#7687) Sebastian Klingler [df30494] Fix amplitude key injection on mobile (#7690) Dylan Jeffers [f8c4f43] [PAY-2513] Fix file size text wrap on web DownloadRow (#7688) Reed [7084823] [PAY-2517] Change name of temp download dir on mobile (#7671) Reed [051d95f] [PROTO-1668] placement for mediorum uploads (#7668) Steve Perkins [17f9536] [C-3857] Remove the shadow on the verified handle section in new sign up flow (#7681) Kyle Shanks [0d6556a] [INF-659] Remove flipper and enable experimental debugger (#7685) Sebastian Klingler [336dc7a] [PROTO-1669] Find Mongo changes when offline and clean up DDEX (#7680) Theo Ilie [043c3a5] [C-3863] Increase touch target of close button of artist preview banner (#7666) Kyle Shanks [b2a0609] [C-3881] Add apiKey to the jwt payload (#7684) Raymond Jacobson [dc41e93] [Discovery] Add year time range for metrics endpoints [C-3753] (#7676) nicoback2 [c104b7e] [C-3854] Update size of social buttons in new sign up flow (#7682) Kyle Shanks [f772ae8] [C-3859] Fix mobile handle validation quirks (#7664) JD Francis [21f5d8e] [DVRL-10] Ensure history uses track activity model (#7678) Raymond Jacobson [3a8b27a] Bump version to 0.6.44 audius-infra [eb858ee] Upgrade dependencies used in alpine base image (#7675) Reed [9824ddc] [PROTO-1670] Find userId in DDEX parser and remove artist allowlist (#7662) Theo Ilie [fc4ead6] ⚠️ [PAY-2421] Upload stems in parallel (#7644) Marcus Pasell [831ca36] [C-3870] Update AccountHeader handle text to be one line (#7672) Kyle Shanks [5c87774] [C-3876] Remove stems icons (#7670) Dylan Jeffers [dded48a] Pin dnspython to version 2.5.0 (#7673) Isaac Solo [ec07ff1] [C-3861] Remove default padding on text inputs for andoird (#7669) Kyle Shanks [3c217da] [C-3873] Migrate to harmony IconButton (#7659) Dylan Jeffers [a8ff559] [PAY-2229] Add lossless UI metrics (#7665) Saliou Diallo [0c37ede] Revert "PROTO-1574: sepolia config and register all nodes" (#7667) alecsavvy [51cdfd6] [DVRL-10] Add auth middleware sdk changes for history (#7663) Raymond Jacobson [4ec621f] Bump version to 0.6.43 audius-infra

This reverts commit fc4ead6.

[skip ci] ## Changelog - 2024-02-29 [b9c440e] Test pt 3 [sliptype] - 2024-02-28 [63494e0] Use turbo to build sdk in release workflow [sliptype] - 2024-02-22 [21f5d8e] [DVRL-10] Ensure history uses track activity model (#7678) [Raymond Jacobson] - 2024-02-21 [fc4ead6] ⚠️ [PAY-2421] Upload stems in parallel (#7644) [Marcus Pasell] - 2024-02-21 [0c37ede] Revert "PROTO-1574: sepolia config and register all nodes" (#7667) [alecsavvy] - 2024-02-21 [51cdfd6] [DVRL-10] Add auth middleware sdk changes for history (#7663) [Raymond Jacobson] - 2024-02-21 [01563db] v1.5.67 [audius-infra] - 2024-02-20 [eaacf44] [PROTO-1667] Add DDEX sessions and admin+artist auth (#7645) [Theo Ilie] - 2024-02-20 [04bb965] PROTO-1574: sepolia config and register all nodes (#7648) [alecsavvy] - 2024-02-15 [fa33251] Add priority fee ixs to purchase txs (#7613) [Reed] - 2024-02-14 [23edd81] ⚠️ [ONC-19] Error on failed gas estimation (#7569) [Raymond Jacobson] - 2024-02-14 [a22dc7a] rework relay selection (#7539) [alecsavvy] - 2024-02-10 [8ecf162] v1.5.66 [audius-infra] - 2024-02-09 [d924e3d] [PAY-2464] Add access to purchases + sales endpoints (#7526) [Reed] - 2024-02-08 [85194b5] force reselection on relay 500s (#7529) [alecsavvy] - 2024-02-08 [328e805] [PAY-2450][PAY-2451] Purchase indexing uses access from memo (#7510) [Reed] - 2024-02-07 [4b39b68] ⚠️ [PAY-2387][PAY-2385] Web/Mobile Web: Change Email and Change Password (#7352) [Marcus Pasell] - 2024-02-05 [e513ff3] [PAY-2329] Use download_conditions in purchase flow (#7385) [Reed] - 2024-02-03 [42966c1] v1.5.65 [audius-infra] - 2024-02-02 [4140f4c] [PAY-2448] Exclude system USDC transactions by default (#7429) [Randy Schott] - 2024-02-01 [ba35a42] [PAY-2373][PAY-2406][PAY-2374] Support file sizes for stems / downloads (#7421) [Raymond Jacobson] - 2024-01-31 [3c898a8] Unbundle @audius/common (#7379) [Dylan Jeffers] - 2024-01-27 [6da6f9a] v1.5.64 [audius-infra] - 2024-01-24 [7b53f91] ⚠️ [INF-547] SSR Track Page (#7213) [Sebastian Klingler] - 2024-01-23 [439f10b] Add ChallengesApi, RewardManager to SDK (#7026) [Marcus Pasell] - 2024-01-23 [17f3a6c] [Web][Libs] QA for Link Audius Profile to Dashboard Wallet feature [C-3683] [C-3686] (#7283) [nicoback2] - 2024-01-20 [6a332ac] v1.5.63 [audius-infra] - 2024-01-19 [1599367] [PROTO-1598] Use tRPC in ddex (#7150) [Theo Ilie] - 2024-01-18 [2a15a64] v1.5.62 (#7243) [Dylan Jeffers] - 2024-01-17 [c3c2148] [Web] [Libs] Disconnect dashboard wallet - web and SDK changes (#7209) [nicoback2] - 2024-01-16 [ab1d3e9] Add USDC gated downloads flow (#6899) [Saliou Diallo] - 2024-01-14 [8d031a4] [PROTO-1599] Add OTP support (#7193) [Raymond Jacobson] - 2024-01-13 [39448b8] v1.5.61 [audius-infra] - 2024-01-12 [6e2927a] repairEntityManagerUserV2 (#7189) [Marcus Pasell] - 2024-01-12 [2b5cc0e] v1.5.60 (#7187) [Raymond Jacobson] - 2024-01-12 [a021a74] [Web][SDK] Send EM transaction from Audius client for connect wallet feature (previously reviewed) (#7186) [nicoback2] - 2024-01-12 [c8efc84] [Discovery] Allow `wallet_signature` message in CreateDashboardWalletUser tx metadata to contain user handle (#7173) [nicoback2] - 2024-01-12 [60652a0] Fix top_listeners restx schema (#7157) [Steve Perkins] - 2024-01-11 [a0a5c01] rest endpoint for track top_listeners (#7139) [Steve Perkins] - 2024-01-09 [274bc9c] Support creating dashboard wallet user in SDK (#7106) [nicoback2] - 2024-01-06 [b9f0b43] v1.5.59 [audius-infra] - 2024-01-02 [7928e12] Improve sdk track upload logs (#7007) [Sebastian Klingler] - 2023-12-30 [8ae982c] v1.5.58 [audius-infra] - 2023-12-29 [8300ad0] [C-3550] Upgrade react native to 0.73.1 (#7042) [Dylan Jeffers] - 2023-12-29 [116a19a] [SDK] Fix validate ETH address util (#7043) [nicoback2] - 2023-12-28 [8c1e557] Reland "Add Solana Support to SDK with sdk.users.sendTip() (#6891)" (#7025) [Marcus Pasell] - 2023-12-28 [858da9f] [SDK] Export DashboardWalletUsersAPI + types in SDK index file, add missing param (#7039) [nicoback2] - 2023-12-28 [a2d87ca] [SDK] Have DashboardWalletUsers extend generated dashboard wallet users API (#7033) [nicoback2] - 2023-12-27 [ad3f877] [C-3547] Upgrade eslint, typescript, and prettier (#7036) [Dylan Jeffers] - 2023-12-27 [dea3dfb] Run comms locally via audius-compose (#7028) [Marcus Pasell] - 2023-12-26 [44b99b6] Re-gen SDK for DashboardWalletUsers APIs (#7027) [nicoback2] - 2023-12-23 [daa5a9b] v1.5.57 [audius-infra] - 2023-12-22 [179ba8a] [SDK] Add dashboard wallet users API to SDK [C-3530] (#6971) [nicoback2] - 2023-12-21 [a002560] Fix sdk Storage retry logic (#7006) [Sebastian Klingler] - 2023-12-21 [500f9a7] Publish sdk and fix publish flow (#6995) [Sebastian Klingler] - 2023-12-20 [9eefe64] Revert "Add Solana Support to SDK with sdk.users.sendTip() (#6891)" (#7002) [Marcus Pasell] - 2023-12-20 [a542274] Add Solana Support to SDK with sdk.users.sendTip() (#6891) [Marcus Pasell] - 2023-12-19 [8967859] Fix sdk write w/ vite and fix npm publish (#6986) [Sebastian Klingler] - 2023-12-16 [7bb95df] v1.5.56 [audius-infra] - 2023-12-15 [7a8d979] Use retry logic for coinflow (#6957) [Marcus Pasell] - 2023-12-13 [b18a138] rm does_follow_current_user from API response (#6836) [Steve Perkins] - 2023-12-11 [7929dd0] [PAY-2128] Implement general coinflow scaffolding (#6847) [Raymond Jacobson] - 2023-12-11 [ab24471] Add memo for purchaser user id (#6846) [Reed] - 2023-12-09 [7e1dbfc] v1.5.55 [audius-infra] - 2023-12-08 [976b787] [PAY-2221] Purchase content through payment router (#6880) [Reed] - 2023-12-07 [f9de2f3] [INF-557] Convert libs to esm so vite HMR works (#6882) [Sebastian Klingler] - 2023-12-05 [9946c48] Payment router instructions in @audius/spl (#6862) [Reed] - 2023-12-05 [cd431cb] Fix typo in libs solana decimals constants (#6863) [Reed] - 2023-12-02 [f3398f6] v1.5.54 [audius-infra] - 2023-11-28 [00b0d25] Add Solana Relay to Discovery (#6782) [Marcus Pasell] - 2023-11-25 [e794c14] v1.5.53 [audius-infra] - 2023-11-20 [db88ed2] Fix electron w/ vite (#6747) [Sebastian Klingler] - 2023-11-18 [32514e5] v1.5.52 [audius-infra] - 2023-11-11 [d4d13e6] v1.5.51 [audius-infra] - 2023-11-09 [62de2d4] Update default RPCs (#6639) [Raymond Jacobson] - 2023-11-06 [fd12b98] [PAY-2091] Update aao error emojis and libs mapped error (#6593) [Saliou Diallo] - 2023-11-06 [6c473bb] ⚠️ [INF-507] Migrate from CRA to Vite (#6567) [Sebastian Klingler] - 2023-11-04 [41a3c27] v1.5.50 [audius-infra] - 2023-10-28 [f5cf244] v1.5.48 [audius-infra] - 2023-10-26 [76b14ce] [PAY-2041] Connect stripe session creation errors to analytics (#6465) [Randy Schott] - 2023-10-25 [3a176a2] Add fallbacks to SDK upload (#5970) [Theo Ilie] - 2023-10-21 [5c4aba5] v1.5.47 [audius-infra] - 2023-10-19 [785a815] [PAY-2060] Fix wrong challenge claimed so far amounts (#6398) [Reed] - 2023-10-18 [492a51a] [INF-484] Remove postinstall builds (#6381) [Sebastian Klingler] - 2023-10-17 [8cc5a64] [PAY-2009] Challenge cooldown UI (#6363) [Reed] - 2023-10-17 [741a0a6] Revert "Revert "[INF-498] Leverage turborepo cache in docker builds i… (#6365) [Sebastian Klingler] - 2023-10-16 [5162831] PROTO-1335: relay upload delay (#6281) [Alec Savvy] - 2023-10-14 [11fd490] v1.5.46 [audius-infra] - 2023-10-13 [5781951] [PAY-1886] Challenge cooldown: created_at migration + backend updates (#6326) [Reed] - 2023-10-12 [d133591] Fix audio balance error C-3207 C-3208 (#6317) [nicoback2] - 2023-10-12 [a384b2f] Revert "[INF-498] Leverage turborepo cache in docker builds in CI (#6… (#6320) [Isaac Solo] - 2023-10-11 [efd5ba7] [INF-498] Leverage turborepo cache in docker builds in CI (#6293) [Sebastian Klingler] - 2023-10-07 [7c00f48] v1.5.45 [audius-infra] - 2023-10-05 [faa348d] @audius/sdk: v3.0.11-beta.21 [audius-infra]

[skip ci] ## Changelog - 2024-02-29 [2738765] Use turbo for sdk release (#7748) [Sebastian Klingler] - 2024-02-28 [c2a9c46] Add ddex_app field for ddex tracks/playlists to discovery (#7734) [Michelle Brier] - 2024-02-26 [398db27] PROTO-1574: sepolia (#7698) [alecsavvy] - 2024-02-26 [beb054b] Remove all usage of PROTOCOL_DIR env var (#7714) [Danny] - 2024-02-24 [5452dcb] v1.5.68 [audius-infra] - 2024-02-22 [21f5d8e] [DVRL-10] Ensure history uses track activity model (#7678) [Raymond Jacobson] - 2024-02-21 [fc4ead6] ⚠️ [PAY-2421] Upload stems in parallel (#7644) [Marcus Pasell] - 2024-02-21 [0c37ede] Revert "PROTO-1574: sepolia config and register all nodes" (#7667) [alecsavvy] - 2024-02-21 [51cdfd6] [DVRL-10] Add auth middleware sdk changes for history (#7663) [Raymond Jacobson] - 2024-02-21 [01563db] v1.5.67 [audius-infra] - 2024-02-20 [eaacf44] [PROTO-1667] Add DDEX sessions and admin+artist auth (#7645) [Theo Ilie] - 2024-02-20 [04bb965] PROTO-1574: sepolia config and register all nodes (#7648) [alecsavvy] - 2024-02-15 [fa33251] Add priority fee ixs to purchase txs (#7613) [Reed] - 2024-02-14 [23edd81] ⚠️ [ONC-19] Error on failed gas estimation (#7569) [Raymond Jacobson] - 2024-02-14 [a22dc7a] rework relay selection (#7539) [alecsavvy] - 2024-02-10 [8ecf162] v1.5.66 [audius-infra] - 2024-02-09 [d924e3d] [PAY-2464] Add access to purchases + sales endpoints (#7526) [Reed] - 2024-02-08 [85194b5] force reselection on relay 500s (#7529) [alecsavvy] - 2024-02-08 [328e805] [PAY-2450][PAY-2451] Purchase indexing uses access from memo (#7510) [Reed] - 2024-02-07 [4b39b68] ⚠️ [PAY-2387][PAY-2385] Web/Mobile Web: Change Email and Change Password (#7352) [Marcus Pasell] - 2024-02-05 [e513ff3] [PAY-2329] Use download_conditions in purchase flow (#7385) [Reed] - 2024-02-03 [42966c1] v1.5.65 [audius-infra] - 2024-02-02 [4140f4c] [PAY-2448] Exclude system USDC transactions by default (#7429) [Randy Schott] - 2024-02-01 [ba35a42] [PAY-2373][PAY-2406][PAY-2374] Support file sizes for stems / downloads (#7421) [Raymond Jacobson] - 2024-01-31 [3c898a8] Unbundle @audius/common (#7379) [Dylan Jeffers] - 2024-01-27 [6da6f9a] v1.5.64 [audius-infra] - 2024-01-24 [7b53f91] ⚠️ [INF-547] SSR Track Page (#7213) [Sebastian Klingler] - 2024-01-23 [439f10b] Add ChallengesApi, RewardManager to SDK (#7026) [Marcus Pasell] - 2024-01-23 [17f3a6c] [Web][Libs] QA for Link Audius Profile to Dashboard Wallet feature [C-3683] [C-3686] (#7283) [nicoback2] - 2024-01-20 [6a332ac] v1.5.63 [audius-infra] - 2024-01-19 [1599367] [PROTO-1598] Use tRPC in ddex (#7150) [Theo Ilie] - 2024-01-18 [2a15a64] v1.5.62 (#7243) [Dylan Jeffers] - 2024-01-17 [c3c2148] [Web] [Libs] Disconnect dashboard wallet - web and SDK changes (#7209) [nicoback2] - 2024-01-16 [ab1d3e9] Add USDC gated downloads flow (#6899) [Saliou Diallo] - 2024-01-14 [8d031a4] [PROTO-1599] Add OTP support (#7193) [Raymond Jacobson] - 2024-01-13 [39448b8] v1.5.61 [audius-infra] - 2024-01-12 [6e2927a] repairEntityManagerUserV2 (#7189) [Marcus Pasell] - 2024-01-12 [2b5cc0e] v1.5.60 (#7187) [Raymond Jacobson] - 2024-01-12 [a021a74] [Web][SDK] Send EM transaction from Audius client for connect wallet feature (previously reviewed) (#7186) [nicoback2] - 2024-01-12 [c8efc84] [Discovery] Allow `wallet_signature` message in CreateDashboardWalletUser tx metadata to contain user handle (#7173) [nicoback2] - 2024-01-12 [60652a0] Fix top_listeners restx schema (#7157) [Steve Perkins] - 2024-01-11 [a0a5c01] rest endpoint for track top_listeners (#7139) [Steve Perkins] - 2024-01-09 [274bc9c] Support creating dashboard wallet user in SDK (#7106) [nicoback2] - 2024-01-06 [b9f0b43] v1.5.59 [audius-infra] - 2024-01-02 [7928e12] Improve sdk track upload logs (#7007) [Sebastian Klingler] - 2023-12-30 [8ae982c] v1.5.58 [audius-infra] - 2023-12-29 [8300ad0] [C-3550] Upgrade react native to 0.73.1 (#7042) [Dylan Jeffers] - 2023-12-29 [116a19a] [SDK] Fix validate ETH address util (#7043) [nicoback2] - 2023-12-28 [8c1e557] Reland "Add Solana Support to SDK with sdk.users.sendTip() (#6891)" (#7025) [Marcus Pasell] - 2023-12-28 [858da9f] [SDK] Export DashboardWalletUsersAPI + types in SDK index file, add missing param (#7039) [nicoback2] - 2023-12-28 [a2d87ca] [SDK] Have DashboardWalletUsers extend generated dashboard wallet users API (#7033) [nicoback2] - 2023-12-27 [ad3f877] [C-3547] Upgrade eslint, typescript, and prettier (#7036) [Dylan Jeffers] - 2023-12-27 [dea3dfb] Run comms locally via audius-compose (#7028) [Marcus Pasell] - 2023-12-26 [44b99b6] Re-gen SDK for DashboardWalletUsers APIs (#7027) [nicoback2] - 2023-12-23 [daa5a9b] v1.5.57 [audius-infra] - 2023-12-22 [179ba8a] [SDK] Add dashboard wallet users API to SDK [C-3530] (#6971) [nicoback2] - 2023-12-21 [a002560] Fix sdk Storage retry logic (#7006) [Sebastian Klingler] - 2023-12-21 [500f9a7] Publish sdk and fix publish flow (#6995) [Sebastian Klingler] - 2023-12-20 [9eefe64] Revert "Add Solana Support to SDK with sdk.users.sendTip() (#6891)" (#7002) [Marcus Pasell] - 2023-12-20 [a542274] Add Solana Support to SDK with sdk.users.sendTip() (#6891) [Marcus Pasell] - 2023-12-19 [8967859] Fix sdk write w/ vite and fix npm publish (#6986) [Sebastian Klingler] - 2023-12-16 [7bb95df] v1.5.56 [audius-infra] - 2023-12-15 [7a8d979] Use retry logic for coinflow (#6957) [Marcus Pasell] - 2023-12-13 [b18a138] rm does_follow_current_user from API response (#6836) [Steve Perkins] - 2023-12-11 [7929dd0] [PAY-2128] Implement general coinflow scaffolding (#6847) [Raymond Jacobson] - 2023-12-11 [ab24471] Add memo for purchaser user id (#6846) [Reed] - 2023-12-09 [7e1dbfc] v1.5.55 [audius-infra] - 2023-12-08 [976b787] [PAY-2221] Purchase content through payment router (#6880) [Reed] - 2023-12-07 [f9de2f3] [INF-557] Convert libs to esm so vite HMR works (#6882) [Sebastian Klingler] - 2023-12-05 [9946c48] Payment router instructions in @audius/spl (#6862) [Reed] - 2023-12-05 [cd431cb] Fix typo in libs solana decimals constants (#6863) [Reed] - 2023-12-02 [f3398f6] v1.5.54 [audius-infra] - 2023-11-28 [00b0d25] Add Solana Relay to Discovery (#6782) [Marcus Pasell] - 2023-11-25 [e794c14] v1.5.53 [audius-infra] - 2023-11-20 [db88ed2] Fix electron w/ vite (#6747) [Sebastian Klingler] - 2023-11-18 [32514e5] v1.5.52 [audius-infra] - 2023-11-11 [d4d13e6] v1.5.51 [audius-infra] - 2023-11-09 [62de2d4] Update default RPCs (#6639) [Raymond Jacobson] - 2023-11-06 [fd12b98] [PAY-2091] Update aao error emojis and libs mapped error (#6593) [Saliou Diallo] - 2023-11-06 [6c473bb] ⚠️ [INF-507] Migrate from CRA to Vite (#6567) [Sebastian Klingler] - 2023-11-04 [41a3c27] v1.5.50 [audius-infra] - 2023-10-28 [f5cf244] v1.5.48 [audius-infra] - 2023-10-26 [76b14ce] [PAY-2041] Connect stripe session creation errors to analytics (#6465) [Randy Schott] - 2023-10-25 [3a176a2] Add fallbacks to SDK upload (#5970) [Theo Ilie] - 2023-10-21 [5c4aba5] v1.5.47 [audius-infra] - 2023-10-19 [785a815] [PAY-2060] Fix wrong challenge claimed so far amounts (#6398) [Reed] - 2023-10-18 [492a51a] [INF-484] Remove postinstall builds (#6381) [Sebastian Klingler] - 2023-10-17 [8cc5a64] [PAY-2009] Challenge cooldown UI (#6363) [Reed] - 2023-10-17 [741a0a6] Revert "Revert "[INF-498] Leverage turborepo cache in docker builds i… (#6365) [Sebastian Klingler] - 2023-10-16 [5162831] PROTO-1335: relay upload delay (#6281) [Alec Savvy] - 2023-10-14 [11fd490] v1.5.46 [audius-infra] - 2023-10-13 [5781951] [PAY-1886] Challenge cooldown: created_at migration + backend updates (#6326) [Reed] - 2023-10-12 [d133591] Fix audio balance error C-3207 C-3208 (#6317) [nicoback2] - 2023-10-12 [a384b2f] Revert "[INF-498] Leverage turborepo cache in docker builds in CI (#6… (#6320) [Isaac Solo] - 2023-10-11 [efd5ba7] [INF-498] Leverage turborepo cache in docker builds in CI (#6293) [Sebastian Klingler] - 2023-10-07 [7c00f48] v1.5.45 [audius-infra] - 2023-10-05 [faa348d] @audius/sdk: v3.0.11-beta.21 [audius-infra]

This reverts commit fc4ead6.

rickyrombo added 11 commits February 15, 2024 20:53

Support uploading stems in parallel for single tracks

af9c0a1

Upload stems in parallel for multiple tracks

06093e8

Handle failed stem parents

6ca6828

Use same handleUpload call to upload both tracks and stems, and remove stems of failed tracks

Better handling of upload errors

906650c

Stops the finish page from spinning forever by letting it treat "error" similarly to "complete", and having the saga mark the track as errored if it fails

Add comment warning

6e247b9

prettier

cc7dad5

every upload is multi upload now

2fc3f04

fix percentage going backwards, fix rounding

075ebd2

always report progress (remove isStem)

ea75ad1

remove unused

44f91e5

better handle errors in multitrack, add toast

0a70756

rickyrombo requested a review from raymondjacobson February 17, 2024 04:57

pull-request-size bot added the size/XL label Feb 17, 2024

raymondjacobson approved these changes Feb 21, 2024

View reviewed changes

rickyrombo added 2 commits February 20, 2024 17:40

publicize generateTrackId

10ad80d

handle error

8076c6d

Merge remote-tracking branch 'origin/main' into mjp-stems-parallel

b455f4d

rickyrombo merged commit fc4ead6 into main Feb 21, 2024
32 checks passed

rickyrombo deleted the mjp-stems-parallel branch February 21, 2024 22:48

rickyrombo added a commit that referenced this pull request Feb 27, 2024

Revert "⚠️ [PAY-2421] Upload stems in parallel (#7644)"

251bd62

This reverts commit fc4ead6.

rickyrombo added a commit that referenced this pull request Feb 27, 2024

Revert "⚠️ [PAY-2421] Upload stems in parallel (#7644)"

d43570a

This reverts commit fc4ead6.

rickyrombo added a commit that referenced this pull request Mar 13, 2024

Revert "⚠️ [PAY-2421] Upload stems in parallel (#7644)"

1f6de01

This reverts commit fc4ead6.

dylanjeffers added a commit that referenced this pull request Mar 19, 2024

Revert "⚠️ [PAY-2421] Upload stems in parallel (#7644)"

a20a33f

This reverts commit fc4ead6.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

⚠️ [PAY-2421] Upload stems in parallel #7644

⚠️ [PAY-2421] Upload stems in parallel #7644

rickyrombo commented Feb 17, 2024

raymondjacobson left a comment

raymondjacobson Feb 21, 2024

rickyrombo Feb 21, 2024

raymondjacobson Feb 21, 2024

rickyrombo Feb 21, 2024

raymondjacobson Feb 21, 2024

raymondjacobson Feb 21, 2024

rickyrombo Feb 21, 2024

raymondjacobson Feb 21, 2024

raymondjacobson Feb 21, 2024

rickyrombo Feb 21, 2024

raymondjacobson Feb 21, 2024

rickyrombo Feb 21, 2024

raymondjacobson Feb 21, 2024

rickyrombo Feb 21, 2024

raymondjacobson Feb 21, 2024

audius-infra commented Feb 21, 2024

gitguardian bot commented Feb 21, 2024

audius-infra commented Feb 21, 2024

⚠️ [PAY-2421] Upload stems in parallel #7644

⚠️ [PAY-2421] Upload stems in parallel #7644

Conversation

rickyrombo commented Feb 17, 2024

Description

How Has This Been Tested?

raymondjacobson left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

audius-infra commented Feb 21, 2024

gitguardian bot commented Feb 21, 2024

⚠️ GitGuardian has uncovered 2 secrets following the scan of your pull request.

audius-infra commented Feb 21, 2024