[PAY-2533, PAY-2534, PAY-2545, PAY-2547, PAY-2532] Index album purchases #7735

amendelsohn · 2024-02-28T20:04:44Z

Description

Enable usdc album purchases 💵

[PAY-2534] Update content type in purchase transaction memo
[PAY-2545] Create album_price_history table
[PAY-2547] Index price history changes on album upload/edit
[PAY-2532] Add album to purchasable content type enum
Add purchase album support to audius-cmd
Integration tests for album purchases via payment_router and user_bank

How Has This Been Tested?

End-to-end test using local stack and audius-cmd
Passing integration tests

gitguardian · 2024-02-28T20:04:54Z

⚠️ GitGuardian has uncovered 24 secrets following the scan of your pull request.

Please consider investigating the findings and remediating the incidents. Failure to do so may lead to compromising the associated services or software components.

🔎 Detected hardcoded secrets in your pull request

GitGuardian id	GitGuardian status	Secret	Commit	Filename
8818104	Triggered	Generic High Entropy Secret	`eeca61e`	packages/embed/.env.prod	View secret
8818104	Triggered	Generic High Entropy Secret	`eeca61e`	packages/embed/.env.dev	View secret
8818104	Triggered	Generic High Entropy Secret	`eeca61e`	packages/embed/.env.stage	View secret
2460751	Triggered	Generic High Entropy Secret	`eeca61e`	protocol-dashboard/.env.prod	View secret
2460750	Triggered	Generic High Entropy Secret	`eeca61e`	protocol-dashboard/.env.prod	View secret
2460750	Triggered	Generic High Entropy Secret	`eeca61e`	protocol-dashboard/.env.prod	View secret
8370413	Triggered	Alchemy API Key	`eeca61e`	protocol-dashboard/.env.prod	View secret
8370413	Triggered	Alchemy API Key	`eeca61e`	protocol-dashboard/.env.prod	View secret
8370414	Triggered	Alchemy API Key	`eeca61e`	protocol-dashboard/.env.prod	View secret
8370414	Triggered	Alchemy API Key	`eeca61e`	protocol-dashboard/.env.prod	View secret
2416684	Triggered	Generic High Entropy Secret	`eeca61e`	protocol-dashboard/.env.stage	View secret
8370415	Triggered	Alchemy API Key	`eeca61e`	protocol-dashboard/.env.prod	View secret
8370415	Triggered	Alchemy API Key	`eeca61e`	protocol-dashboard/.env.prod	View secret
2416686	Triggered	Generic High Entropy Secret	`eeca61e`	protocol-dashboard/.env.stage	View secret
8370416	Triggered	Alchemy API Key	`eeca61e`	protocol-dashboard/.env.prod	View secret
8370416	Triggered	Alchemy API Key	`eeca61e`	protocol-dashboard/.env.prod	View secret
688750	Triggered	Generic High Entropy Secret	`eeca61e`	protocol-dashboard/.env.stage	View secret
688750	Triggered	Generic High Entropy Secret	`eeca61e`	protocol-dashboard/.env.prod	View secret
688750	Triggered	Generic High Entropy Secret	`eeca61e`	protocol-dashboard/.env.stage	View secret
688750	Triggered	Generic High Entropy Secret	`eeca61e`	protocol-dashboard/.env.prod	View secret
2460749	Triggered	Generic High Entropy Secret	`eeca61e`	protocol-dashboard/.env.prod	View secret
2460749	Triggered	Generic High Entropy Secret	`eeca61e`	protocol-dashboard/.env.prod	View secret
2416685	Triggered	Generic High Entropy Secret	`eeca61e`	protocol-dashboard/.env.stage	View secret
9349019	Triggered	Generic High Entropy Secret	`89f4e39`	dev-tools/config.json	View secret

🛠 Guidelines to remediate hardcoded secrets

Understand the implications of revoking this secret by investigating where it is used in your code.
Replace and store your secrets safely. Learn here the best practices.
Revoke and rotate these secrets.
If possible, rewrite git history. Rewriting git history is not a trivial act. You might completely break other contributing developers' workflow and you risk accidentally deleting legitimate data.

To avoid such incidents in the future consider

following these best practices for managing and storing secrets including API keys and other credentials
install secret detection on pre-commit to catch secret before it leaves your machine and ease remediation.

^{🦉 GitGuardian detects secrets in your source code to help developers and security teams secure the modern development process. You are seeing this because you or someone else with access to this repository has authorized GitGuardian to scan your pull request.

Our GitHub checks need improvements? Share your feedbacks!}

audius-infra · 2024-02-28T20:29:23Z

Preview this change https://demo.audius.co/am-purchaseable-album-type

schottra

Looks good! Have some minor suggestions and am curious what the plan is for download gating. But generally seems good to go.

schottra · 2024-02-29T15:56:04Z

packages/common/src/api/purchases.ts

@@ -84,6 +84,7 @@ const purchasesApi = createApi({
            context
          )
        }
+        // TODO: Purchaseable Albums - fetch metadata for albums


Usually like to attach linear IDs/URLs to these so they don't get lost.

schottra · 2024-02-29T15:58:52Z

packages/discovery-provider/integration_tests/tasks/test_index_payment_router.py

@@ -160,6 +171,56 @@ def test_process_payment_router_tx_details_valid_purchase(app):
        assert transaction_record.tx_metadata == str(trackBuyerId)


+def test_process_payment_router_tx_details_valid_purchase_album(app):


I know these tests are cumbersome to write and I would love to see us exercise more than just the golden path here (not paying enough to one of the recipients for instance)

so the non-golden paths are getting exercised in existing tests for track purchases, eg. validating that the shape of stream_conditions is what we expect, validating splits etc. since the logic is pretty much the same we figured it was sufficient just to test that the golden path for albums works as expected.

schottra · 2024-02-29T16:00:01Z

packages/discovery-provider/integration_tests/tasks/test_index_payment_router.py

+            .first()
+        )
+        assert purchase is not None
+        assert purchase.seller_user_id == 1


Might recommend using the constants above (trackOwnerId etc) to make these assertions clear to future readers. Though that probably also means changing them to contentOwnerId and so on.

schottra · 2024-02-29T16:01:16Z

packages/discovery-provider/integration_tests/tasks/test_index_user_bank.py

@@ -216,6 +227,78 @@ def test_process_user_bank_tx_details_valid_purchase(app):
        assert buyer_transaction_record.tx_metadata == str(track_owner_id)


+def test_process_user_bank_tx_details_valid_purchase_album(app):


Oof I wish we had just migrated existing balance purchases to go through payment router so you didn't have to update both indexers.

schottra · 2024-02-29T16:01:44Z

packages/discovery-provider/integration_tests/tasks/test_index_user_bank.py

+            .first()
+        )
+        assert purchase is not None
+        assert purchase.seller_user_id == 1


Same suggestions for this indexer's tests as for the payment router tests.

schottra · 2024-02-29T16:03:36Z

packages/discovery-provider/integration_tests/tasks/user_bank_mock_transactions.py

@@ -51,7 +51,7 @@
 # Pubkey.find_program_address([bytes(USDC_MINT)], Pubkey.from_string(CLAIMABLE_TOKENS_PDA))
 USDC_PDA = "7vKR1WSmyHvBmCvKPZBiN66PHZqYQbXw51SZdwtVd9Dt"

-# contentType:contentId:blockNumber
+# contentType:contentId:blockNumber:purchaseUserId:accessType


This comment feels slightly misleading as most of the memos below don't include accessType?

schottra · 2024-02-29T16:04:33Z

packages/discovery-provider/src/models/playlists/album_price_history.py

+    __tablename__ = "album_price_history"
+
+    playlist_id = Column(Integer, nullable=False, primary_key=True)
+    splits = Column(JSONB(), nullable=False)


I wonder if it makes sense for us to add doc comments about the exact structure of said JSONB?

schottra · 2024-02-29T16:05:48Z

packages/discovery-provider/src/tasks/entity_manager/entities/playlist.py

+):
+    """Adds an entry in the track price history table to record the price change of a track or change of splits if necessary."""
+    new_record = None
+    is_stream_gated = playlist_metadata.get(


Just curious if this is supposed to intersect with the download-gating logic? I'm not up to speed on the feature spec here, just wondering if we're supposed to support purchasable downloads and if this code feels set up to add that extension without a lot of refactoring.

we're not supporting download-gated albums at the moment!

schottra · 2024-02-29T16:07:29Z

packages/discovery-provider/src/tasks/entity_manager/entities/playlist.py

+            if "splits" in usdc_purchase:
+                splits = usdc_purchase["splits"]
+                # TODO: better validation of splits
+                if isinstance(splits, dict):


Do we have any tests around this indexer? Might be nice to validate that we correctly reject invalid fields.

adding follow-up ticket PAY-2553 to add this validation and test it

schottra · 2024-02-29T16:11:29Z

packages/libs/src/services/schemaValidator/schemas/playlistSchema.json

+        "splits": {
+          "type": "object",
+          "patternProperties": {
+            "^[1-9A-HJ-NP-Za-km-z]{32,44}$": {


Going to assume at first glance that this is a well-known regex for base58 solana addresses?

yea looks like it. Again copied from the other test data 😅

rickyrombo

+1 to Randy's review!

audius-infra · 2024-02-29T19:29:03Z

Preview this change https://demo.audius.co/am-purchaseable-album-type

[73c35e5] Bump mobile app version for full-app release (#7770) Dylan Jeffers [fa9c7ce] [PAY-2490] Delete backfill txs tables (#7768) Reed [acf5840] Fix reset-password-modal icon + ui (#7769) Dylan Jeffers [cb9c022] [C-3929] Fix select-artist genres (#7765) Dylan Jeffers [709ff70] Fix reset password (#7767) Marcus Pasell [1c1474e] [C-3708] Fix wallet text clipping issue in withdrawals table (#7762) Kyle Shanks [7f4b187] Add an isNativeMobile prop to amplitude (#7766) JD Francis [4f6972f] [PAY-2527] Return `playlists_containing_track` from track apis (#7764) Andrew Mendelsohn [dbc6dc1] Fix download buttons erroneously displaying (#7763) Reed [8cf2464] [PAY-2531] Grant access to stream track if user has purchased album (#7753) Reed [762a9ac] [C-3926] Add error helper text to new sign in password field (#7761) Kyle Shanks [6d6117f] [C-3885] Migrate to harmony text-link (#7746) Dylan Jeffers [82dd8c5] Update dapp-store build artifacts audius-infra [1c554c6] [C-3924] Fix misc font/type issues on purchase flow (#7760) Raymond Jacobson [b695203] [C-3923] Remove double badges on purchase flow (#7759) Raymond Jacobson [e1dd6e0] [PAY-2488] Remove stripe option from purchase flow (#7755) Raymond Jacobson [4931acb] Bump version to 0.6.50 audius-infra [c8eaa02] [C-2928] Add auth request signing middleware to sdk (#7724) Raymond Jacobson [0d2d3b1] Add verifier checks to discovery relay (#7756) Raymond Jacobson [c3f0a99] useHarmonyField (#7514) Marcus Pasell [ce226e7] Publish harmony to npm (#7751) Sebastian Klingler [d5c0c98] [C-3897] Fix ai-attribution dropdown (#7754) Dylan Jeffers [26f9c7d] [PAY-2533] Index album purchases (#7735) Andrew Mendelsohn [375f5b5] [C-3888] Add header-shadow to sign-up-screen (#7750) Dylan Jeffers [dbbbc88] @audius/sdk: v3.0.35 audius-infra [5125501] [DRVL-16] - Documentation Site Updates (#7699) Sam Gutentag [2738765] Use turbo for sdk release (#7748) Sebastian Klingler [23072cd] [C-3918] Fix wallet and dev-apps icons (#7749) Dylan Jeffers [a4bbe1c] Update stem limit to 20 (#7745) Raymond Jacobson [f4863c6] Fix social sign ups redirecting unintentionally (#7744) JD Francis [557142b] Disable play button if all collection tracks deleted (#7743) Saliou Diallo [5f28d5d] Add others to skip otp (#7741) Raymond Jacobson [b33bc5b] Add download gated column (#7742) Saliou Diallo [9f0f84e] Bump version to 0.6.49 audius-infra [c2a9c46] Add ddex_app field for ddex tracks/playlists to discovery (#7734) Michelle Brier [c932495] [PROTO-1686] Setup DDEX choreography infra and related improvements (#7739) Theo Ilie [aec57f9] Misc sign up social fixes (#7737) JD Francis [682a384] [ONC-35] Fix crash on android due to react native importing images as numbers (#7738) Marcus Pasell [273be01] Misc Sign up fixes (spacing & page swipe animation) (#7736) JD Francis [cb39083] [ONC-34] Fix bug in mobile suggested collection tracks (#7731) Saliou Diallo [8db7400] [C-3866] Update placement of theme reset in mobile sign out to make colors display properly (#7728) Kyle Shanks [0d15ae7] [C-3898] Fix faded gray screen android on sign up (#7730) JD Francis [a0d314b] Bump version to 0.6.48 audius-infra [f43e956] [PROTO-1513] Add e2e ddex test (#7733) Theo Ilie [f5ad4c3] Fix harmony ssr (#7732) Dylan Jeffers [06af9d2] [C-3909] Fix upload track challenge copy (#7729) Saliou Diallo [8f73c19] [C-3899] Fix android able to go back on signup (#7727) JD Francis [d04b4bc] [PAY-2529] Index album stream_conditions (#7720) Reed [8966f88] [QA-779][QA-704] Fix some UI issues (#7725) Saliou Diallo [d29c83f] Misc Sign Up QA Fixes (#7722) JD Francis [90b098a] Discovery placement hosts (#7718) Steve Perkins [9674780] Bump version to 0.6.47 audius-infra [398db27] PROTO-1574: sepolia (#7698) alecsavvy [40583e6] Improve search analytics (#7696) Isaac Solo [3e1b544] Add env linter (#7697) Raymond Jacobson [5a3498c] [DVRL-11] Move client history requests to sdk (#7679) Raymond Jacobson [beb054b] Remove all usage of PROTOCOL_DIR env var (#7714) Danny [6230eca] [PAY-2528] is_stream_gated + stream_conditions on Playlist (#7717) Reed [4136985] [PAY-2524, PAY-2525, PAY-2526] Add collections_containing_track field to tracks (#7708) Andrew Mendelsohn [f25cc23] Fix the same typo in another spot (#7716) Theo Ilie [f51ccd8] [C-3858] Additional padding on select artists screen (#7715) JD Francis [9ff7262] [C-3858] Social sign on loading screen adjustments (#7710) JD Francis [4c3a35c] Fix typo breaking audius-compose push (#7713) Theo Ilie [74facea] Bump mobile versions for prod release (#7712) Dylan Jeffers [392bb21] Bump version to 0.6.46 audius-infra

[skip ci] ## Changelog - 2024-03-11 [f1d70a2] [INF-663] Fix external package versioning in monorepo (#7783) [Sebastian Klingler] - 2024-03-09 [51ec3ab] v1.5.71 [audius-infra] - 2024-03-02 [fb7f377] v1.5.70 [audius-infra] - 2024-03-02 [43d0fea] v1.5.69 [audius-infra] - 2024-03-01 [fa9c7ce] [PAY-2490] Delete backfill txs tables (#7768) [Reed] - 2024-02-29 [c8eaa02] [C-2928] Add auth request signing middleware to sdk (#7724) [Raymond Jacobson] - 2024-02-29 [26f9c7d] [PAY-2533] Index album purchases (#7735) [Andrew Mendelsohn] - 2024-02-29 [dbbbc88] @audius/sdk: v3.0.35 [audius-infra]

amendelsohn added 13 commits February 26, 2024 16:01

add album purchase type

1e8eea8

create album_price_history_table

72353fd

index_purchase changes

b9cdd86

update price history table on playlist change

5257a8f

Merge branch 'main' into am-purchaseable-album-type

eeca61e

update test script for album purchase

b1ce711

fix import

7bcece7

pipe stream_conditions through to db; add cmd for premium album

aec3439

working purchase script

a877407

add purchase album test

774e23f

add payment_router album purchase test

115830f

remove debug logs

b08c9c7

type for purchase content type

abd358c

amendelsohn requested review from schottra, rickyrombo, isaacsolo, dylanjeffers and sddioulde February 28, 2024 20:04

amendelsohn assigned dharit-tan Feb 28, 2024

pull-request-size bot added the size/XXL label Feb 28, 2024

schottra approved these changes Feb 29, 2024

View reviewed changes

rickyrombo approved these changes Feb 29, 2024

View reviewed changes

amendelsohn added 3 commits February 29, 2024 10:49

Merge branch 'main' into am-purchaseable-album-type

89f4e39

Merge branch 'main' into am-purchaseable-album-type

9fe83d1

PR feedback

4e893fa

amendelsohn merged commit 26f9c7d into main Feb 29, 2024
38 checks passed

amendelsohn deleted the am-purchaseable-album-type branch February 29, 2024 21:30

amendelsohn changed the title ~~[PAY-2533] Index album purchases~~ [PAY-2533, PAY-2534, PAY-2545, PAY-2547, PAY-2532] Index album purchases Feb 29, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[PAY-2533, PAY-2534, PAY-2545, PAY-2547, PAY-2532] Index album purchases #7735

[PAY-2533, PAY-2534, PAY-2545, PAY-2547, PAY-2532] Index album purchases #7735

amendelsohn commented Feb 28, 2024

gitguardian bot commented Feb 28, 2024 •

edited

Loading

audius-infra commented Feb 28, 2024

schottra left a comment

schottra Feb 29, 2024

schottra Feb 29, 2024

dharit-tan Feb 29, 2024

schottra Feb 29, 2024

schottra Feb 29, 2024

schottra Feb 29, 2024

schottra Feb 29, 2024

schottra Feb 29, 2024

schottra Feb 29, 2024

dharit-tan Feb 29, 2024

schottra Feb 29, 2024

amendelsohn Feb 29, 2024

schottra Feb 29, 2024

amendelsohn Feb 29, 2024

rickyrombo left a comment

audius-infra commented Feb 29, 2024

		@@ -160,6 +171,56 @@ def test_process_payment_router_tx_details_valid_purchase(app):
		assert transaction_record.tx_metadata == str(trackBuyerId)


		def test_process_payment_router_tx_details_valid_purchase_album(app):

		@@ -216,6 +227,78 @@ def test_process_user_bank_tx_details_valid_purchase(app):
		assert buyer_transaction_record.tx_metadata == str(track_owner_id)


		def test_process_user_bank_tx_details_valid_purchase_album(app):

[PAY-2533, PAY-2534, PAY-2545, PAY-2547, PAY-2532] Index album purchases #7735

[PAY-2533, PAY-2534, PAY-2545, PAY-2547, PAY-2532] Index album purchases #7735

Conversation

amendelsohn commented Feb 28, 2024

Description

How Has This Been Tested?

gitguardian bot commented Feb 28, 2024 • edited Loading

⚠️ GitGuardian has uncovered 24 secrets following the scan of your pull request.

audius-infra commented Feb 28, 2024

schottra left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rickyrombo left a comment

Choose a reason for hiding this comment

audius-infra commented Feb 29, 2024

gitguardian bot commented Feb 28, 2024 •

edited

Loading