Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[PROTO-1622] Implement core parsing for DDEX ERN v3 releases #7538

Merged
merged 3 commits into from
Feb 9, 2024

Conversation

theoilie
Copy link
Contributor

@theoilie theoilie commented Feb 9, 2024

Description

Implements the core logic to support NewReleaseMessage from the DDEX ERN v3 spec for both tracks and albums.

  • Uses xmlquery (as opposed to marshalling the whole XML file or streaming it) to maintain flexibility for variations between versions and to provide informative error messages
  • There are many TODOs which I'll iterate on and clean up. This is a good stopping point because it fully parses an XML file into Audius-recognizable fields and is starting to grow too large to review comfortably

How Has This Been Tested?

This parses out an example XML file into the expected structs for uploading to Audius. A followup will add tests and make it write to the Mongo collection for the publisher module to use (edit: done).

Copy link
Contributor

@michellebrier michellebrier left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice!

@theoilie theoilie merged commit 1d795c8 into main Feb 9, 2024
9 checks passed
@theoilie theoilie deleted the theo-proto1622-ddex-parser branch February 9, 2024 22:07
audius-infra pushed a commit that referenced this pull request Feb 10, 2024
[6a3a1e4] Support linking in gnosis safe profile info (#7550) Raymond Jacobson
[ca9b814] [PAY-2430][PAY-2433] Fix android download logic (#7547) Randy Schott
[79fa06c] [DVRL-10] Add auth_middleware to history route (#7540) Raymond Jacobson
[edb21ea] [C-3802] Match Avatar styles more closely to prod (#7546) Sebastian Klingler
[1d795c8] [PROTO-1622] Implement core parsing for DDEX ERN v3 releases (#7538) Theo Ilie
[ec379ba] fix action button error (#7545) Andrew Mendelsohn
[15d6102] Mobile: Change Email, Change Password CR Fixes (#7543) Marcus Pasell
[0c47c73] Fix lint (#7544) Raymond Jacobson
[d924e3d] [PAY-2464] Add access to purchases + sales endpoints (#7526) Reed
[3264c59] update timings and set dryrun to false on cron (#7542) alecsavvy
[960c625] [DVRL-14] - Integrate Google Analytics in Docs (#7541) Sam Gutentag
[cf0c985] [C-3786, C-3787] Fix album link casing; Fix required artwork (#7536) Andrew Mendelsohn
[b4cc968] Bump version to 0.6.34 audius-infra
[85194b5] force reselection on relay 500s (#7529) alecsavvy
[3e3a7fa] run pg_migrate.sh test in CI (#7475) Steve Perkins
[51af96c] [PAY-2454] Mobile: Change Email, Change Password (#7505) Marcus Pasell
[60a30c3] [C-3775] Fix co-sign placement (#7535) Dylan Jeffers
[177ca1c] [C-3748] Address harmony callouts (#7534) Dylan Jeffers
[3d621f1] [C-3783, C-3778] Fix collection screen messages (#7531) Andrew Mendelsohn
[26ddbf3] [Web] Fix some weird arrow sizes [C-3801] (#7533) nicoback2
[24defc5] Fix red icons + badge hover color in popup menu [C-3799] [C-3798] (#7532) nicoback2
[225fe2c] [PAY-2473] Change 'original' copy to 'lossless' in download section (#7524) Reed
[94967d5] [PAY-2466] Update  banner copy in purchase flow (#7523) Reed
[bb662db] [PAY-2477] Remove blue background from purchased in download section (#7522) Reed
[80a170c] [PAY-2426] Fix native harmony text and buttons (#7528) Dylan Jeffers
[fb49bc6] [Web] Fix more icon issues after Harmony migration C-3789 (#7515) nicoback2
[97a0839] Skip jupiter tx in user bank withdrawal indexing (#7527) Raymond Jacobson
[ea6169a] Fix Withdrawal flow errors due to slippage calculations (#7525) Randy Schott
[1e5f66b] [C-3644] Fix sign-up tests (#7282) Dylan Jeffers
[1e99ac8] More DDEX cleanup (#7516) Michelle Brier
[cf0042b] [PAY-2405] Add missing orig file cid and file name migration (#7393) Saliou Diallo
[b4522da] [ONC-15] Fix artist dashboard plays bucketing (#7519) Sebastian Klingler
[4eb06b0] openresty config (#7517) Steve Perkins
[328e805] [PAY-2450][PAY-2451] Purchase indexing uses access from memo (#7510) Reed
[fd2a0a8] Fix parser (#7518) Raymond Jacobson
[b72d30b] [PAY-2434][PAY-2427] Fix miscellaneous lossless issues (#7508) Saliou Diallo
[0730584] [PAY-2460] Original file sizes (#7513) Raymond Jacobson
[9b71ca4] Bump version to 0.6.33 audius-infra
[45bd451] [DVRL-8] - adding code workspace file and plugin configs (#7481) Sam Gutentag
[3c10459] [PAY-2423] Render download options in equal-sized tabs (#7507) Randy Schott
[3fd97ab] [PAY-2432][PAY-2462] Cancellable downloads (#7512) Raymond Jacobson
[5eff264] Misc DDEX cleanup + publisher sets delivery status (#7506) Michelle Brier
[0f2024c] [C-3780] Fix track history api call (#7511) Andrew Mendelsohn
[4b71006] Update DN README.md for testing (#7509) Reed
[f636ee3] [DVRL-9] - upgrade docusaurus to v3.1.1 and add mermaid diagrams (#7498) Sam Gutentag
[d61803f] Migrate token-amount-input to harmony (#7500) Dylan Jeffers
[686eadc] [C-3773] Migrate radio from stems to harmony (#7487) Dylan Jeffers
[152b1f4] [PAY-2398] Support stems mobile web (#7504) Raymond Jacobson
[c092b7c] [ONC-10] Fix styles for ArtistRecommendations pictures (#7503) Sebastian Klingler
[8711074] [PAY-2461] Create track_price_history row for download-gated-only tracks (#7502) Reed
[ed8b9f9] [PAY-2431][PAY-2416] Fix misc web UI items (#7489) Raymond Jacobson
[1994ce4] Revert "Disable usdc bank split in dev mode (#6300)" (#7501) Reed
[3fc17fc] [Web] Fix upload button icon color and dropdown input caret size [C-3771] (#7484) nicoback2
[4b39b68] ⚠️ [PAY-2387][PAY-2385] Web/Mobile Web: Change Email and Change Password (#7352) Marcus Pasell
[ed95055] Remove stems segmented-conrol, fix description (#7499) Dylan Jeffers
[e935693] Integrate ddex into CI auto-upgrade (#7496) Theo Ilie
[10b89f9] Fix tag search navigating on web (#7497) Isaac Solo
[c633c27] [PAY-2449] USDC purchase access migration (#7460) Reed
[e13220b] [PAY-2415][PAY-2417][PAY-2419][PAY-2420][PAY-2437] Fix lossless issues (#7483) Saliou Diallo
[abc0d1e] [PAY-2456][PAY-2438][PAY-2435] Bring mobile purchase flow up to lossless spec (#7490) Raymond Jacobson
[a202295] Miscellaneous data fetching fixes for Pay and Earn page (#7493) Randy Schott
[ffd3a6f] Add correct type to manual transfer confirmation switch field (#7491) Randy Schott
[6b0e0aa] [DVRL-7] - update frontmatter and switch to manual sidebars (#7477) Sam Gutentag
[62688b8] Fix search personalization in client (#7495) Isaac Solo
[f148992] [PAY-2365] Update logic to access nft gated tracks (#7471) Saliou Diallo
[ed94cb8] Add missing cors package (#7494) Theo Ilie
[11ca5fa] [PAY-2457] Fix bad conditional for showing transaction link (#7492) Randy Schott
[1e6af0e] Bump version to 0.6.32 audius-infra
[4a49749] [PAY-2353] Adds offramp analytics (#7480) Randy Schott
[bff53fa] [PAY-2444] Title case stems (#7478) Raymond Jacobson
[de70135] [PROTO-1621] Implement ddex indexer (#7486) Theo Ilie
[de047ad] Improve user search on exact matches (#7476) Isaac Solo
[2c659ff] Update crawler regex to include all crawlers (#7485) Sebastian Klingler
[fa30731] Delete broken or unused release/publish workflows (#7482) Danny
[33c620c] fix ddex ingester CI (#7479) Michelle Brier
[d4726c7] Enable SSR (#7473) Sebastian Klingler
[26ffd2f] Fix SSR deploy cache (#7474) Sebastian Klingler
[ae4edfe] fix audius-cmd && replicate relay locally (#7462) alecsavvy
[ba54eb2] [PAY-2408][PAY-2443][PAY-2397] Bring web purchase up to lossless spec (#7469) Raymond Jacobson
[0934d73] DDEX crawler (#7470) Michelle Brier
[9e23123] [Web] Fix OAuth red icons [C-3766] (#7472) nicoback2
[42e2e7b] Bump version to 0.6.31 audius-infra
[087f38e] [PAY-2429] Add hitslop to stem download row (#7458) Raymond Jacobson
[b8f6e7a] [Harmony] [Web] Fix red icons and playback icons [C-3765] (#7468) nicoback2
[9e0131e] [C-3749] Migrate stem components to harmony round 1 (#7459) Dylan Jeffers
[39e4575] [C-3763] Restrict harmony imports in mobile (#7464) Dylan Jeffers
[3aed8bc] [C-3764] Use history in handleClickRoute pt 2 (#7467) Sebastian Klingler
[546a04f] Fixes signon saga type (#7466) Dylan Jeffers
[01ccbb2] [C-3764] Use history in handleClickRoute (#7465) Sebastian Klingler
[4f87a53] [C-3762] Unblock stage deploy by uploading sourcemaps to s3 (#7461) Sebastian Klingler
[64466e7] [INF-654] Fix mobile SSR render (#7463) Sebastian Klingler
[8e02b23] [C-3665] - Prevent back on select genres page (#7447) JD Francis
[80debdf] [C-3746] & [C-3745] Fix CoverPhoto and ProfileImage not showing correctly in native sign up  (#7451) JD Francis
[ad583bc] Fix electric subgenre select-artist requests (#7327) Dylan Jeffers
[8bfbe14] Migrate selectable pills to harmony (#7416) Dylan Jeffers
[f59fb87] [C-3607] Store & restore profile image and cover photo between new sign up pages (#7455) JD Francis
[e513ff3] [PAY-2329] Use download_conditions in purchase flow (#7385) Reed
[c792f8b] [PAY-2352][PAY-2396] Fix transactions table state after withdrawal (#7457) Randy Schott
[9d75ad2] Fix race condition when initting sdk in ddex (#7456) Theo Ilie
[9f169e7] DDEX indexer -> publisher talking (#7452) Michelle Brier
[109be52] [PAY-2404] Display correct file extensions when selecting download quality (#7446) Reed
[4939341] [PAY-2436] Add isOwner purchaseable downloads copy (#7454) Raymond Jacobson
[f5f9d37] [C-3741] Fix sign up blocker when having no artists due to local stack (#7440) JD Francis
[f0e5810] Fix cosign SVG fill color not using fill prop (#7448) JD Francis
[87df3c3] Bump version to 0.6.30 audius-infra
[8b97390] [Web] [Harmony] Harmony Web Icons Migration (#7392) nicoback2
[b7cf7b9] [Native] Fix OTP paste on mobile [C-3692] (#7367) nicoback2
[965f243] [ONC-7] Move to opensea api v2 (#7449) Raymond Jacobson
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants