CIP-0048? | NFT metadata references and payloads #249
Conversation
made the specification section more concise
|
Discussion and example use cases can be found on the Cardano forum. |
KtorZ
changed the title
|
I would mark this as an informational CIP as it recommends a practice within the current 721 metadata which users are free to ignore or implement as willing. |
|
@Jack-0 not to worry: the term I followed the Cardano Forum thread about this but didn't take any sides. For other editors: the URI link for this proposal (also in comment above) probably at this time has a complete discussion of all arguments for & against adoption. Most of the reactions are that "it's not necessary" somehow but that seems to be refuted by a statements like "It's my Tx data and I'm paying for it, so I can do what I want with it." I would just like the major points in this thread to also be here in this Github discussion so they can be easily referenced at the CIP meetings, to make sure this PR doesn't remain stalled. So @Jack-0 could you please digest those points here along with whatever rebuttals you have (maybe also include each of these in the CIP draft itself if not already)? A good way to progress this would be to attend the CIP meeting on Discord in which CIP advocates & authors can get the mic, time permitting, and always comment in the text stream... a better platform than Crowdcast which we had at the time you submitted this PR 🤓 https://discord.gg/GhfMM9bf |
|
Yes I would be happy to. I'll include key points here asap. |
|
Major points mentioned on the forum post: A blockchain like Cardano is not for storing jpegs or to archive data files: How do you distinguish between two or more very similar 721s metadata: Is it more flexible or more generic than existing standards for NFT meta data: Could type be implicitly inferred from value: Is it backwards compatible: More compact than CBOR: Future compatibility: Personal note: How this CIP can also be used to reduce chain data: Opensource query example |
|
I also was just glancing over the 'Smart NFTs' proposal #263. |
There was a problem hiding this comment.
I don't think this CIP currently meets the required quality we would demand of a CIP, and I would strongly suggest to spend more time polishing it on various aspects:
- I found quite many typos or confusing sentences; I fixed a few but there are more left.
- There are many "holes" in the specification at the moment which would lead to potentially many different incompatible implementations. Such a standard should be very precise on how the data is specified and interpreted. Few recommendations:
- Avoid double-notations (e.g.
referencesorr; chose one); - Avoid references to other similar systems in the specification itself (e.g. mention of
ext2) unless you clearly point to another specification from which you re-use pieces. It's fine to mention as part of the rationale or abstract, but in the specification it can be quite confusing especially if the other system is only vaguely referred to. - Define terms and operations that are ambiguous (e.g. 'concatenate')
- Be consistent in the wording being used, don't use two different words to refer to the same concept (e.g.
pointersvslinks). - Avoid doubts / open-questions (e.g. "not sure if..."), or clearly mark them as "TODO" to make them points to address.
- Avoid double-notations (e.g.
CIP-0048/CIP-0048.md
Outdated
| * Provide a mechanism to reference on chain data stored in a different policy | ||
|
|
||
| # Specification | ||
| Please note I offer optional single char names to allow futher reduction in metadata size if desired |
There was a problem hiding this comment.
Avoid first-person voice in CIPs.
| Please note I offer optional single char names to allow futher reduction in metadata size if desired | |
| Please note that some tags can be specified as single char names to allow further reduction in metadata size if desired. |
CIP-0048/CIP-0048.md
Outdated
| |---|---| --- | | ||
| |references, refs, r | contains data tag and src tag | ```'r':{ "data:"text/plain","src":[0,1] }```| | ||
| |payload, p | contains raw data | ```'p':{ "0":["h","e","l","l","o"], "1":"world" }``` | | ||
| |version| this is described in 721 | not sure if it should be version 1.1? instead of 2| |
There was a problem hiding this comment.
Since this is an optional extension of the standard, it wouldn't make sense to use the same versioning scheme as CIP-0025 because there might be other proposed extensions which either conflict or are simply used in total separation from this one. Say we have two mutually exclusive extensions for CIP-0025 with versions 2.1 and 2.2. Now, shouldn't 2.2 implies that 2.1 is also used? What if, no, people just want to use 2.1?
I think CIP-0025 lacks extensibility mechanisms; something that can let new extensions be specified in an additive manner (e.g. an extra field extensions: ['CIP-0048']).
CIP-0048/CIP-0048.md
Outdated
| |version| this is described in 721 | not sure if it should be version 1.1? instead of 2| | ||
|
|
||
| *inside the refences tag* | ||
| |tags that are reseved inside references |description | example |
There was a problem hiding this comment.
| |tags that are reseved inside references |description | example | |
| |tags that are reserved inside references |description | example |
CIP-0048/CIP-0048.md
Outdated
| |tags that are reseved inside references |description | example | ||
| |---|---| --- | | ||
| |data| contains the data type MIME | for example if base 64, data:"text/plain;base64"| | ||
| |src|contains a list of poitners to payloads | ```"src":[0,1]``` or ``` "src":["payload0","payload1"]```| |
There was a problem hiding this comment.
| |src|contains a list of poitners to payloads | ```"src":[0,1]``` or ``` "src":["payload0","payload1"]```| | |
| |src|contains a list of pointers to payloads | ```"src":[0,1]``` or ``` "src":["payload0","payload1"]```| |
CIP-0048/CIP-0048.md
Outdated
| |policy, p | references another policy. If none, assume current policy | ```'r':{"data:"text/plain","src":{[0,1]},"p":"<POLICY_ID>}"```| | ||
| |txhash, tx | references a specific tx hash (optional) | ```'r':{"data:"text/plain","src":{[0,1]},"tx":"<TX_HASH>}"```| | ||
| |type | the format/type of the data inside the payload |```'r':{"data:"image/jpeg","src":{[0,1]},"type":"ipfs"```| | ||
| |b, back, backup | fallback, if a link to the data is used in the 'src' tag and is broken revert to a fallback link. | ```'r':{"src":{[0,1]},"b":{[2,3]}} ``` or for multiple fallbacks use ```'r':{"src":{[0,1]},"b":"{'0':{[2,3]}, '1'{[4,5]} }"``` | |
CIP-0048/CIP-0048.md
Outdated
| |src|contains a list of poitners to payloads | ```"src":[0,1]``` or ``` "src":["payload0","payload1"]```| | ||
| |policy, p | references another policy. If none, assume current policy | ```'r':{"data:"text/plain","src":{[0,1]},"p":"<POLICY_ID>}"```| | ||
| |txhash, tx | references a specific tx hash (optional) | ```'r':{"data:"text/plain","src":{[0,1]},"tx":"<TX_HASH>}"```| | ||
| |type | the format/type of the data inside the payload |```'r':{"data:"image/jpeg","src":{[0,1]},"type":"ipfs"```| |
There was a problem hiding this comment.
What difference is made between data and type?
CIP-0048/CIP-0048.md
Outdated
| |utf-8| data is directly in payload as utf-8 | | ||
| |b64 | data is in base 64 encoded in payload | | ||
| |reference? | advance feature, points to a reference | | ||
| |none| assume data is raw and using the defined media type | |
There was a problem hiding this comment.
It seems like each of those types has a precise semantic, what is it?
CIP-0048/CIP-0048.md
Outdated
| 2. Check the version tag | ||
| 3. If a payload is found append that to an map or some data structure | ||
| 4. Given a nft find there references | ||
| 5. Concatenate the data for all given references |
There was a problem hiding this comment.
How is "concatenate" defined here? What if the data isn't text or bytes?
CIP-0048/CIP-0048.md
Outdated
| ``` | ||
| # Retrieve payload information using references | ||
| 1. Find all transactions with the 721 label | ||
| 2. Check the version tag |
CIP-0048/CIP-0048.md
Outdated
| } | ||
| ``` | ||
| # Retrieve payload information using references | ||
| 1. Find all transactions with the 721 label |
There was a problem hiding this comment.
What is the scope of research here? A block? The whole chain? Should only payload under the same policy be considered?
rphair
added
the
State: Waiting for Author
rphair
removed
the
State: Waiting for Author
|
@Jack-0 a naming collision for CIP-0049 with #250 came up at this week's CIP meeting so we've removed the pending CIP number from #343. Consequently in your last comment, so it provides a proper index for readers without confusing that reference, can you please also remove CIP-0049 from the heading? (e.g. simply "NFT metadata extension tag") |
|
I have done that, assuming nothing needs changing with this CIP. If for some reason we decide that #343 is not appropriate then this CIP would not use such a feature and it would just be yet another CIP that defines optional properties. I was hoping 343 would bring some clarity to NFT optional metadata properties. |
|
that's all fine... I was just asking (to avoid confusion in the repository) if you would please remove the terms CIP-0049 from this comment, where it currently says:
and
|
|
Would also like to mention CIP 68 as it would be worth considering Reference and Payload usage in the Datum metadata standard |
|
@Jack-0 in the last hour's CIP meeting we discussed (mainly with @KtorZ) how we might finally get this one out of the queue. Though it would otherwise be in a state where it could be merged as a document for NFT developers to "take it or leave it" (since there is no documented support from existing projects or NFT marketplaces), it seems this is dependent on #343 which in turn requires modifications to CIP-0025. So for now we can only proceed once @alessandrokonrad @SmaugPool can comment here and/or in #343 about how they feel about adding support for the NFT metadata extension tag. And @Jack-0 I think I've asked this question before (when you were separating this PR from #343) but still don't know if it's possible that this mechanism could work without that extension tag.... if so please indicate that clearly here & we then might be free to go ahead and finally merge this. |
There was a problem hiding this comment.
@Jack-0 likewise with #343 (review) I'm marking this Waiting for Author with the intent to close as superseded by the pending CIP-0088 (also inviting that author to contribute to the discussion here).
We can keep this alive if someone can point out there is any feature provided, or use case handled, which is not addressed in CIP-0088. Otherwise with no other supportive action taken we will probably close this soon.
rphair
added
the
State: Waiting for Author
KtorZ
added
Category: Tokens
rphair
added
the
State: Likely Deprecated
|
In reply to #249 (comment) This proposal that adds support for linking multiple nfts via references and payloads will work without the |
| ## Important note | ||
|
|
||
| This CIP mentions the `ext` json key defined by [CIP49](https://github.com/cardano-foundation/CIPs/pull/343/files) which is an outcome of work on CIP48. |
There was a problem hiding this comment.
This section may be unnecessary if we transition to CIP-88 to support a "signal" that these references are in use.
| # Specification | ||
|
|
||
| ### **Inside the 721 metadata JSON** | ||
|
|
||
| | reserved | description | scope | | ||
| | -------- | ------------------------------------------------------ | ------------------------- | | ||
| | refs | references tag, defines where to find the onchain data | Inside `<asset_name>` tag | | ||
| | p | contains raw data (on chain data). | Inside `<policy_id>` tag | |
There was a problem hiding this comment.
I believe that we can eliminate both the refs and p definitions within individual token metadata here if we switch to a URI-schema to signal support by way of CIP-88. More details to be shown below.
| ## General structure | ||
|
|
||
| ```json | ||
| { | ||
| "721": { | ||
| "ext": ["cip48"], // tells 'user' cip48 is in use within the metadata | ||
| "<policy_id>": { | ||
| "<asset_name>": { | ||
| "project": "<string>", | ||
| "name": "<string>", | ||
| "image": "<ref_name | uri | array>" // image is defined by refs or a uri | ||
| // references | ||
| "refs": [ | ||
| { | ||
| "name": "<ref_name>", | ||
| "mediaType": "text/plain", | ||
| "src": ["<payload_id>", "<payload_id>"] // array of required payload ids (ordered) | ||
| } | ||
| ] | ||
| }, | ||
| // payload | ||
| "p": { | ||
| "<payload_id>": ["<data>", "<data>"], // payload0 | ||
| "<payload_id>": ["<data>"] // payload1 | ||
| } | ||
| } | ||
| } | ||
| } | ||
| ``` |
There was a problem hiding this comment.
I think the best way to embed references in existing metadata structures would be via a structured URI format as defined in the spec. Rather than just using a name and forcing all parsers to scan for references a defined URI schema would make it easier to detect.
URI Format
uriSchema = {
scheme : string .size (4..64) ; Examples: ar://, ipfs://, etc
path * : string .size (1..64) ; Examples: [ar/ipfs hash]
}
Hypothetical CIP-48 URI format:
uriSchema = ["cip48://","referenceA"]
In this way a downstream parser may quickly and easily identify a CIP-48-specific reference only on fields that match the format.
There was a problem hiding this comment.
@Crypto2099 do I understand correctly, with this new approach, that there would be no need for the corresponding proposal that has long accompanied this one (which details the external metadata references)?
There was a problem hiding this comment.
Yes, I believe the OP and myself agree that the ext flag would be made redundant if CIP-88 were implemented as both have the same function of signaling intent to support a further extension to CIP-25.
There was a problem hiding this comment.
great then, I'll remove the Likely Deprecated tag from this CIP and leave it on #343.
| ### **Optional `type` tag for `refs`** | ||
|
|
||
| The default CIP-48 behavior is to assume the assets parent `<policy_id>` contains all the payloads. | ||
|
|
||
| Optionally we define `type` and two options `policy` and `txhash`. To allow for non-default behavior | ||
|
|
||
| - `policy` | ||
| - Defines an external policy (instead of the default behavior of using the parent policy) | ||
| - example | ||
| ```json | ||
| "refs": [ | ||
| { | ||
| "name": "...", | ||
| "mediaType": "...", | ||
| "type": { | ||
| "policy": "<some_external_policy>" | ||
| } | ||
| } | ||
| ] | ||
| ``` | ||
| - `txhash` | ||
| - Defines an array of transaction hashes (instead of the default behavior of using the parent policy) | ||
| - example | ||
| ```json | ||
| "refs": [ | ||
| { | ||
| "name": "...", | ||
| "mediaType": "...", | ||
| "type": { | ||
| "txhash": ["<mint_txhash>", "<mint_txhash>"] | ||
| } | ||
| } | ||
| ] | ||
| ``` | ||
| - `txhash` is faster than parsing all transactions in a given policy. | ||
| - `txhash` can dramatically reduce onchain data usage if duplicated data is referenced here _(however, the bytes required to define the reference would have to be less than the reference data itself)_ |
There was a problem hiding this comment.
If we presume that the majority of references could be declared at a top-level utilizing CIP-88, this is just some rough thoughts but perhaps we could expand the references "type" to optionally point to another on-chain pointer or to a direct payload.
{
48: {
0: '1.0.0', ; Version?
1: [ ; Array of references
{
"name": "referenceA", ; Name of the reference
"type": "pointer | payload", ; Maybe an enum of accepted options?
"payload": { ; Example of a direct payload reference
"mediaType": "text/plain",
"content": "My Cool NFT Project"
},
"payload": { ; Example of a tx pointer reference?
"type": "tx",
"content": "[txhash]",
"context": "metadata" | "datum"
},
"payload": { ; Example of a token point reference?
"type": "token",
"content": [(policy_id),(asset_id)],
"context": "metadata" | "datum"
}
}
]
}
}
There was a problem hiding this comment.
@Jack-0 we talked about moving this forward at the CIP meeting today and I summarised that you & @Crypto2099 as of 3 weeks ago were working on making this compatible with CIP-88 / #467.
This review is to clean up the formatting as per CIP-0001 which was updated since last year, so we can merge it once you & @Crypto2099 can confirm here the two CIPs are compatible... so we also need you to address his #249 (review). (cc @KtorZ)
| @@ -0,0 +1,249 @@ | |||
| --- | |||
| CIP: 48? | |||
There was a problem hiding this comment.
| CIP: 48? | |
| CIP: 48 |
| @@ -0,0 +1,249 @@ | |||
| --- | |||
| CIP: 48? | |||
| Title: 721 References and Payloads | |||
There was a problem hiding this comment.
| Title: 721 References and Payloads | |
| Title: NFT metadata references and payloads |
For consistency with PR title.
| --- | ||
| CIP: 48? | ||
| Title: 721 References and Payloads | ||
| Authors: Jack (ada9000, https://forum.cardano.org/u/jack7e/, @ada9000_ twitter) |
There was a problem hiding this comment.
| Authors: Jack (ada9000, https://forum.cardano.org/u/jack7e/, @ada9000_ twitter) | |
| Authors: | |
| - Jack <some@real.email.example.org> |
There's no means to embed URLs in header text, which makes social links impossible. Even if getting an anonymous email to use here, we need something that looks like an email address if only so our web site builders work properly. This is always a list even if having only one member.
| CIP: 48? | ||
| Title: 721 References and Payloads | ||
| Authors: Jack (ada9000, https://forum.cardano.org/u/jack7e/, @ada9000_ twitter) | ||
| Comments-URI: https://forum.cardano.org/t/cip-extended-721/ |
There was a problem hiding this comment.
| Comments-URI: https://forum.cardano.org/t/cip-extended-721/ | |
| Discussions: | |
| - https://forum.cardano.org/t/cip-extended-721 | |
| - https://github.com/cardano-foundation/CIPs/pull/249 |
| Title: 721 References and Payloads | ||
| Authors: Jack (ada9000, https://forum.cardano.org/u/jack7e/, @ada9000_ twitter) | ||
| Comments-URI: https://forum.cardano.org/t/cip-extended-721/ | ||
| Status: Draft |
There was a problem hiding this comment.
| Status: Draft | |
| Status: Proposed |
| } | ||
| ``` | ||
|
|
||
| ## Notes |
There was a problem hiding this comment.
| ## Notes | |
| ### Notes |
| - The `image` tag can still be used as a thumbnail if required. | ||
| - If there are multiple `refs` defined the references that are _NOT_ defined in the `image` tag have the same behavior as the `files` tag currently defined in CIP25 | ||
|
|
||
| # Example |
There was a problem hiding this comment.
| # Example | |
| ### Example |
| # Backwards compatibility | ||
|
|
||
| Handled via the use of the `"ext"` tag defined in CIP-48. | ||
|
|
There was a problem hiding this comment.
| # Backwards compatibility | |
| Handled via the use of the `"ext"` tag defined in CIP-48. |
I think we've established that the tag will not be used, right? However compatibility would be handled is a good item to included in the Rationale section which still needs to be defined below.
| # Further considerations | ||
|
|
||
| - Later we may want to use different reference types, maybe even plutus contract references. Due to this I have defined the `type` tag inside the `refs` (references) tag. | ||
| - `p` is used instead of **payload** to reduce bytes. Specifically `p` is not used within the `<asset_name>` tag therefore readability is less of a priority than data size. Unlike `refs` which is defined in the `<asset_name>` tag | ||
| - JSON is kept over CBOR or alternatives as this CIP is encapsulated in the 721 metadatum tag and JSON is easier for developers to adopt | ||
|
|
||
| ### Duplicate data | ||
|
|
||
| There could be issues with duplicate payloads. To solve the payload defined in the most recently minted tx takes priority. |
There was a problem hiding this comment.
| # Further considerations | |
| - Later we may want to use different reference types, maybe even plutus contract references. Due to this I have defined the `type` tag inside the `refs` (references) tag. | |
| - `p` is used instead of **payload** to reduce bytes. Specifically `p` is not used within the `<asset_name>` tag therefore readability is less of a priority than data size. Unlike `refs` which is defined in the `<asset_name>` tag | |
| - JSON is kept over CBOR or alternatives as this CIP is encapsulated in the 721 metadatum tag and JSON is easier for developers to adopt | |
| ### Duplicate data | |
| There could be issues with duplicate payloads. To solve the payload defined in the most recently minted tx takes priority. | |
| ## Rationale: how does this CIP achieve its goals? | |
| ### Further considerations | |
| - Later we may want to use different reference types, maybe even plutus contract references. Due to this I have defined the `type` tag inside the `refs` (references) tag. | |
| - `p` is used instead of **payload** to reduce bytes. Specifically `p` is not used within the `<asset_name>` tag therefore readability is less of a priority than data size. Unlike `refs` which is defined in the `<asset_name>` tag | |
| - JSON is kept over CBOR or alternatives as this CIP is encapsulated in the 721 metadatum tag and JSON is easier for developers to adopt | |
| ### Duplicate data | |
| There could be issues with duplicate payloads. To solve the payload defined in the most recently minted tx takes priority. |
Just sketching out a Rationale from the points already included here... maybe also some of what you already wrote above regarding your design decisions.
Especially I think in the Rationale section you should include:
- why you originally solved this problem with the extra tag, and
- why you abandoned that idea when CIP88 came along.
| # Further considerations | ||
|
|
||
| - Later we may want to use different reference types, maybe even plutus contract references. Due to this I have defined the `type` tag inside the `refs` (references) tag. | ||
| - `p` is used instead of **payload** to reduce bytes. Specifically `p` is not used within the `<asset_name>` tag therefore readability is less of a priority than data size. Unlike `refs` which is defined in the `<asset_name>` tag | ||
| - JSON is kept over CBOR or alternatives as this CIP is encapsulated in the 721 metadatum tag and JSON is easier for developers to adopt | ||
|
|
||
| ### Duplicate data | ||
|
|
||
| There could be issues with duplicate payloads. To solve the payload defined in the most recently minted tx takes priority. |
There was a problem hiding this comment.
Finally (last section) include a copyright notice of your choice as defined here: https://github.com/cardano-foundation/CIPs/blob/master/CIP-0001/README.md#licensing
|
|
||
| ### Duplicate data | ||
|
|
||
| There could be issues with duplicate payloads. To solve the payload defined in the most recently minted tx takes priority. |
There was a problem hiding this comment.
| There could be issues with duplicate payloads. To solve the payload defined in the most recently minted tx takes priority. | |
| There could be issues with duplicate payloads. To solve the payload defined in the most recently minted tx takes priority. | |
| ## Path to Active |
As per https://github.com/cardano-foundation/CIPs/blob/master/CIP-0001/README.md#path-to-active. I don't understand the industry well enough to advise on this, but if @Jack-0 you're not sure of acceptance or commercial implementation targets then maybe @Crypto2099 could suggest some.
|
@Jack-0 after review at the CIP meeting today, following last month of inactivity with no response to last review trying to get the document itself in line with existing CIP standards, and without @Crypto2099 providing a bridge to CIP88 which is itself moving forward, this is again marked for deprecation and we intend to close it soon. |
rphair
added
the
State: Likely Deprecated
|
Final call for any reservations at CIP meeting today regarding deprecating this & the related PR (#343 (comment)). @Jack-0 please address the last review if you decide to pursue this further. |
|
For anyone following up, some current progress in token indirection (to store greater amounts of on-chain data and consolidate common elements such as rendering and software) being considered here: #530 |
Introduces methods to store and reference 721 metadata. For details and questions please contact me here or on the Cardano Forums
see rendered Markdown