feat: add `store/get` and `upload/get` capabilities #942

olizilla · 2023-10-04T11:40:46Z

Capability definitions to allow clients to check for inclusion of cid in store shards or upload roots. Also allows client to fetch the additional properties we store. This will, for example, let a client verify all shards for an upload have been removed before removing an upload.

I'm aiming to simplify things with this PR so that we return insertedAt info on the Upload/Store ListItem types, and return the same object shape for a single object get. If this is pleasing, then I will continue in this direction and make Remove also return the same shape if it removes something

Depends on approval of the new capabilities in storacha/specs#82

TODO:

spec update feat: add store/get & upload/get capabilities specs#82
tests

License: MIT

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

These capabilities are intended for checking inclusion and allowing users to verify all the shards for an upload have been removed when removing an upload. see also: storacha/w3up#942 License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

olizilla · 2023-10-17T13:49:31Z

packages/capabilities/src/types.ts

+export type Store = InferInvokedCapability<typeof StoreCaps.store>
+export type StoreAdd = InferInvokedCapability<typeof StoreCaps.add>
+export type StoreGet = InferInvokedCapability<typeof StoreCaps.get>
+export type StoreRemove = InferInvokedCapability<typeof StoreCaps.remove>
+export type StoreList = InferInvokedCapability<typeof StoreCaps.list>


Update to pull each cap off of StoreCaps to be consistent with all the other caps in this file.

olizilla · 2023-10-17T13:50:50Z

packages/capabilities/src/types.ts

@@ -271,13 +283,21 @@ export interface StoreListItem {
  link: UnknownLink
  size: number
  origin?: UnknownLink
+  insertedAt: string


add insertedAt to StoreListItem as we already want to return this info for list usage, and it means we can re-use the type for both list and get operations.

olizilla · 2023-10-17T13:54:20Z

packages/capabilities/src/types.ts

+  insertedAt: string
+  updatedAt: string


Add insertedAt and updatedAt props to UploadListItem. We want to share a timestamp with the user about when this upload was added, and these are the props we have. updatedAt currently means "when did we last add a shard to this upload", while insertedAt means "when did you first add a given root cid as an upload to this space."

olizilla · 2023-10-17T13:55:56Z

packages/upload-api/src/types.ts

-  inspect: (link: UnknownLink) => Promise<StoreGetOk>
+  inspect: (link: UnknownLink) => Promise<StoreInspectOk>


Updated the response type for inspect to be StoreInspectOk as StoreGet was confusing... and even more so now I'm adding a StoreGet.

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

olizilla · 2023-10-17T14:02:28Z

packages/upload-api/test/handlers/store.js

-    const item = await context.testStoreTable.get(spaceDid, link)
+    const item = await context.storeTable.get(spaceDid, link)


storeTable.get is now a thing, so we no longer need the testStoreTable.

olizilla · 2023-10-17T14:02:55Z

packages/upload-api/test/handlers/store.js

-        space: item.space,
        link: item.link.toString(),
        size: item.size,
-        issuer: item.issuer,


i don't think we need to test things here that are not part of the contract

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

packages/capabilities/src/store.js

alanshaw · 2023-10-17T15:55:09Z

packages/capabilities/src/store.js

+    /**
+     * shard CID to fetch info about.
+     */
+    link: Link.optional(),


I wonder if this should be derivable from store/add (similar to space/info)? i.e. if you were able to store/add you should be able to derive store/get capability for the same link.

https://github.com/web3-storage/ucanto/blob/43ea497ca2200f9e1e15dbbdd033ec24895edb7d/packages/interface/src/capability.ts#L122-L126

Perhaps not, write does not imply read...just thinking out loud.

is interesting point, but i'm gonna leave it separate for now.

packages/capabilities/src/types.ts

packages/upload-api/src/store/get.js

packages/upload-api/src/types.ts

packages/upload-api/src/upload/get.js

Co-authored-by: Alan Shaw <alan.shaw@protocol.ai>

…otFound License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

alanshaw

LGTM

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

🤖 I have created a release *beep* *boop* --- ## [10.2.0](capabilities-v10.1.0...capabilities-v10.2.0) (2023-10-19) ### Features * add `store/get` and `upload/get` capabilities ([#942](#942)) ([40c79eb](40c79eb)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

🤖 I have created a release *beep* *boop* --- ## [10.1.0](upload-client-v10.0.1...upload-client-v10.1.0) (2023-10-20) ### Features * add `store/get` and `upload/get` capabilities ([#942](#942)) ([40c79eb](40c79eb)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

🤖 I have created a release *beep* *boop* --- ## [9.2.0](w3up-client-v9.1.0...w3up-client-v9.2.0) (2023-10-20) ### Features * add `store/get` and `upload/get` capabilities ([#942](#942)) ([40c79eb](40c79eb)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: Travis Vachon <travis.vachon@protocol.ai>

🤖 I have created a release *beep* *boop* --- ## [6.2.0](upload-api-v6.1.0...upload-api-v6.2.0) (2023-10-20) ### Features * add `store/get` and `upload/get` capabilities ([#942](#942)) ([40c79eb](40c79eb)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: Oli Evans <oli@protocol.ai>

These capabilities are intended for checking inclusion, or getting the additional properties we store. This will allow clients to verify all the shards for an upload have been removed before removing an upload. ## `store/get` delegation ```js { can: "store/get", with: "did:key:abc...", nb: { link: "bag...", } } ``` ### Response Error if `link` shard cid is not in space or... ```ts interface StoreListItem { /** CID of the stored CAR. */ link: string /** Size in bytes of the stored CAR */ size: number /** Link to a related CAR, used for sharded uploads */ origin?: string, /** ISO-8601 timestamp when CAR was added to the space */ insertedAt: string, } ``` ## `upload/get` delegation ```js { can: "upload/get", with: "did:key:abc...", nb: { root: "bafyabc..." } } ``` ### Response `Error` if root is not in uploads for space or... ```ts interface UploadListItem { /** Root CID of the uploaded data item. */ root: string /** CIDs of CARs that contain the complete DAG for the uploaded data item. */ shards: string[] /** ISO-8601 timestamp when the upload was added to the space. */ insertedAt: string, /** ISO-8601 timestamp when the upload entry was last modified. */ updatedAt: string, } ``` --- see also: storacha/w3up#942 License: MIT --------- Signed-off-by: Oli Evans <oli@protocol.ai>

olizilla added 2 commits October 4, 2023 12:40

feat: add store/get and upload/get capabilities

e9af60b

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

chore: format

87a4c74

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

olizilla mentioned this pull request Oct 4, 2023

feat: add store/get & upload/get capabilities storacha/specs#82

Merged

olizilla added 4 commits October 13, 2023 16:02

feat: store/get & upload/get capabilities

8e6459c

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

Merge remote-tracking branch 'origin/main' into get-caps

1d70111

chore: add new caps to abilitiesAsStrings

abb0058

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

chore: fix tests

17a093a

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

olizilla commented Oct 17, 2023

View reviewed changes

olizilla added 2 commits October 17, 2023 14:57

chore: make diff less noisy

5eb25f3

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

chore: make diff less noisy

a47a118

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

olizilla commented Oct 17, 2023

View reviewed changes

chore: make diff nice

df55473

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

olizilla marked this pull request as ready for review October 17, 2023 14:15

olizilla requested review from alanshaw and vasco-santos October 17, 2023 14:15

alanshaw requested changes Oct 17, 2023

View reviewed changes

olizilla and others added 4 commits October 18, 2023 09:57

chore: apply suggestions from code review

9d8fece

Co-authored-by: Alan Shaw <alan.shaw@protocol.ai>

chore: rename *inpect success types and fix error name for StoreItemN…

c268076

…otFound License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

Merge remote-tracking branch 'origin/main' into get-caps

c8c6bdc

chore: am i not pretty enough?

6927e81

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

alanshaw approved these changes Oct 18, 2023

View reviewed changes

olizilla added 2 commits October 19, 2023 11:08

chore: simplify return type from store table

6544496

License: MIT Signed-off-by: Oli Evans <oli@protocol.ai>

Merge remote-tracking branch 'origin/main' into get-caps

0cfafe1

olizilla merged commit 40c79eb into main Oct 19, 2023
8 checks passed

olizilla deleted the get-caps branch October 19, 2023 10:29

This was referenced Oct 19, 2023

chore(main): release capabilities 10.2.0 #983

Merged

chore(main): release w3up-client 9.2.0 #984

Merged

chore(main): release upload-api 6.2.0 #985

Merged

chore(main): release upload-client 10.1.0 #986

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add `store/get` and `upload/get` capabilities #942

feat: add `store/get` and `upload/get` capabilities #942

olizilla commented Oct 4, 2023 •

edited

Loading

olizilla Oct 17, 2023

olizilla Oct 17, 2023

olizilla Oct 17, 2023

olizilla Oct 17, 2023

olizilla Oct 17, 2023

olizilla Oct 17, 2023

alanshaw Oct 17, 2023

olizilla Oct 18, 2023

alanshaw left a comment

		inspect: (link: UnknownLink) => Promise<StoreGetOk>
		inspect: (link: UnknownLink) => Promise<StoreInspectOk>

		const item = await context.testStoreTable.get(spaceDid, link)
		const item = await context.storeTable.get(spaceDid, link)

feat: add store/get and upload/get capabilities #942

feat: add store/get and upload/get capabilities #942

Conversation

olizilla commented Oct 4, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

alanshaw left a comment

Choose a reason for hiding this comment

feat: add `store/get` and `upload/get` capabilities #942

feat: add `store/get` and `upload/get` capabilities #942

olizilla commented Oct 4, 2023 •

edited

Loading