Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: introduce new administrative capabilities #832

Merged
merged 36 commits into from
Aug 9, 2023
Merged
Show file tree
Hide file tree
Changes from 25 commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
51bf9aa
feat: introduce new administrative capabilities
travis Jul 18, 2023
56d1db1
feat: port blocking logic to this implementation
travis Jul 18, 2023
ba3cfcf
feat: add in-memory implementation of rate limits storage
travis Jul 18, 2023
340ccd1
feat: add types for new capability service definitions
travis Jul 26, 2023
2d1133a
feat: add new capability types to service definition
travis Jul 26, 2023
d2766fa
feat: two more typing tweaks
travis Jul 26, 2023
23793d5
feat: small naming tweak
travis Jul 26, 2023
15576fb
fix: get types checking and a simple test implemented
travis Jul 27, 2023
a92702f
Merge branch 'main' into feat/rate-limits
travis Jul 27, 2023
58714b2
chore: pnpm format
travis Jul 27, 2023
ef9042c
fix: more type updates
travis Jul 27, 2023
4979330
feat: implement new ProvisionsStorage methods
travis Jul 27, 2023
fcee024
feat: limit `rate-limit/remove` to a single ID
travis Jul 27, 2023
05ba4f7
feat: move areAnyBlocked to a helper function
travis Jul 27, 2023
a83335b
fix: fix bug caught in manual testing
travis Jul 27, 2023
333ae9f
feat: invoke Space.allocate in Store.add
travis Jul 27, 2023
0cd9397
fix: fix logic bug in the new `areAnyBlocked` utility function
travis Jul 27, 2023
44a28df
chore: add tests for new capabilities
travis Aug 1, 2023
fe1d5fc
chore: add tests for rate limit invocation handlers
travis Aug 1, 2023
c2e6ea5
feat: remove access verifier and get tests running
travis Aug 2, 2023
1be4e8b
chore: minor cleanup from self-review
travis Aug 2, 2023
80fbbae
chore: make code more prettier
travis Aug 2, 2023
1142f36
feat: add test suite for RateLimitsStorage implementations
travis Aug 3, 2023
e34e206
fix: use context.id rather than context.signer
travis Aug 4, 2023
6c0a7a3
Merge branch 'main' into feat/rate-limits
travis Aug 8, 2023
6103531
Update packages/capabilities/src/rate-limit.js
travis Aug 9, 2023
5c77c44
Update packages/capabilities/src/rate-limit.js
travis Aug 9, 2023
eb8a827
Update packages/capabilities/src/rate-limit.js
travis Aug 9, 2023
9e76024
Merge branch 'main' into feat/rate-limits
travis Aug 9, 2023
944eb4b
feat: add tests for rate limit capability escalation
travis Aug 9, 2023
dd23341
fix: add comments for blocking behavior
travis Aug 9, 2023
261a2a4
fix: stylistic tweaks from PR
travis Aug 9, 2023
6526b9f
feat: tweak rate limit checker semantics
travis Aug 9, 2023
362a698
fix: use prettier to make the code more prettier
travis Aug 9, 2023
1575b92
chore: add comment explaining RateLimitID
travis Aug 9, 2023
22b7a69
fix: per PR feedback use allocate in upload/add for consistency
travis Aug 9, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 11 additions & 5 deletions packages/aggregate-api/src/aggregate/offer.js
Original file line number Diff line number Diff line change
@@ -1,6 +1,10 @@
import * as Server from '@ucanto/server'
import { CBOR } from '@ucanto/core'
import { Node, Piece, Aggregate as AggregateBuilder } from '@web3-storage/data-segment'
import {
Node,
Piece,
Aggregate as AggregateBuilder,
} from '@web3-storage/data-segment'
import * as Aggregate from '@web3-storage/capabilities/aggregate'
import * as Offer from '@web3-storage/capabilities/offer'
import * as API from '../types.js'
Expand Down Expand Up @@ -62,10 +66,12 @@ export const claim = async (
// Validate commP of commPs
const aggregateBuild = AggregateBuilder.build({
size: aggregateSize,
pieces: offers.map(offer => Piece.fromJSON({
height: offer.height,
link: { '/': offer.link.toString() }
}))
pieces: offers.map((offer) =>
Piece.fromJSON({
height: offer.height,
link: { '/': offer.link.toString() },
})
),
})
if (!aggregateBuild.link.equals(piece.link)) {
return {
Expand Down
18 changes: 18 additions & 0 deletions packages/capabilities/src/consumer.js
Original file line number Diff line number Diff line change
Expand Up @@ -24,3 +24,21 @@ export const has = capability({
)
},
})

/**
* Capability can be invoked by a provider to get information about a consumer.
*/
export const get = capability({
can: 'consumer/get',
with: ProviderDID,
nb: struct({
consumer: SpaceDID,
}),
derives: (child, parent) => {
return (
and(equalWith(child, parent)) ||
and(equal(child.nb.consumer, parent.nb.consumer, 'consumer')) ||
ok({})
)
},
})
11 changes: 11 additions & 0 deletions packages/capabilities/src/index.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ import * as Customer from './customer.js'
import * as Console from './console.js'
import * as Offer from './offer.js'
import * as Aggregate from './aggregate.js'
import * as RateLimit from './rate-limit.js'
import * as Subscription from './subscription.js'

export {
Access,
Expand All @@ -26,6 +28,8 @@ export {
Utils,
Aggregate,
Offer,
RateLimit,
Subscription,
}

/** @type {import('./types.js').AbilitiesArray} */
Expand All @@ -52,4 +56,11 @@ export const abilitiesAsStrings = [
Aggregate.offer.can,
Aggregate.get.can,
Offer.arrange.can,
Customer.get.can,
Consumer.has.can,
Consumer.get.can,
Subscription.get.can,
RateLimit.add.can,
RateLimit.remove.can,
RateLimit.list.can,
]
71 changes: 71 additions & 0 deletions packages/capabilities/src/rate-limit.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
/**
* Rate Limit Capabilities
*
* These can be imported directly with:
* ```js
* import * as RateLimit from '@web3-storage/capabilities/rate-limit'
* ```
*
* @module
*/
import { capability, DID, struct, Schema, ok } from '@ucanto/validator'
import { equalWith, and, equal } from './utils.js'

// e.g. did:web:web3.storage or did:web:staging.web3.storage
export const Provider = DID

/**
* Capability can be invoked by an agent to add a rate limit to a subject.
travis marked this conversation as resolved.
Show resolved Hide resolved
*/
export const add = capability({
can: 'rate-limit/add',
with: Provider,
nb: struct({
subject: Schema.string(),
rate: Schema.number(),
}),
derives: (child, parent) => {
return (
and(equalWith(child, parent)) ||
and(equal(child.nb.subject, parent.nb.subject, 'subject')) ||
and(equal(child.nb.rate, parent.nb.rate, 'rate')) ||
ok({})
)
},
})

/**
* Capability can be invoked by an agent to remove rate limits from a subject.
travis marked this conversation as resolved.
Show resolved Hide resolved
*/
export const remove = capability({
can: 'rate-limit/remove',
with: Provider,
nb: struct({
id: Schema.string(),
}),
derives: (child, parent) => {
return (
and(equalWith(child, parent)) ||
and(equal(child.nb.id, parent.nb.id, 'id')) ||
ok({})
)
},
})

/**
* Capability can be invoked by an agent to list rate limits on a subject
travis marked this conversation as resolved.
Show resolved Hide resolved
*/
export const list = capability({
can: 'rate-limit/list',
with: Provider,
nb: struct({
subject: Schema.string(),
}),
derives: (child, parent) => {
return (
and(equalWith(child, parent)) ||
and(equal(child.nb.subject, parent.nb.subject, 'subject')) ||
ok({})
)
},
})
23 changes: 23 additions & 0 deletions packages/capabilities/src/subscription.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
import { capability, DID, struct, ok, Schema } from '@ucanto/validator'
import { equalWith, and, equal } from './utils.js'

// e.g. did:web:web3.storage or did:web:staging.web3.storage
export const ProviderDID = DID.match({ method: 'web' })

/**
* Capability can be invoked by a provider to get information about a subscription.
*/
export const get = capability({
can: 'subscription/get',
with: ProviderDID,
nb: struct({
subscription: Schema.string(),
}),
derives: (child, parent) => {
return (
and(equalWith(child, parent)) ||
and(equal(child.nb.subscription, parent.nb.subscription, 'consumer')) ||
ok({})
)
},
})
92 changes: 90 additions & 2 deletions packages/capabilities/src/types.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
import type { TupleToUnion } from 'type-fest'
import * as Ucanto from '@ucanto/interface'
import type { Schema } from '@ucanto/core'
import { InferInvokedCapability, Unit, DID } from '@ucanto/interface'
import { InferInvokedCapability, Unit, DID, DIDKey } from '@ucanto/interface'
import type { PieceLink } from '@web3-storage/data-segment'
import { space, info, recover, recoverValidation } from './space.js'
import * as provider from './provider.js'
Expand All @@ -12,8 +12,15 @@ import { claim, redeem } from './voucher.js'
import * as AccessCaps from './access.js'
import * as AggregateCaps from './aggregate.js'
import * as OfferCaps from './offer.js'
import * as CustomerCaps from './customer.js'
import * as ConsumerCaps from './consumer.js'
import * as SubscriptionCaps from './subscription.js'
import * as RateLimitCaps from './rate-limit.js'

export type { Unit }

export type AccountDID = DID<'mailto'>

/**
* failure due to a resource not having enough storage capacity.
*/
Expand All @@ -22,6 +29,11 @@ export interface InsufficientStorage {
message: string
}

export interface UnknownProvider extends Ucanto.Failure {
name: 'UnknownProvider'
did: DID
}

export type PieceLinkSchema = Schema.Schema<PieceLink>

// Access
Expand Down Expand Up @@ -68,6 +80,75 @@ export interface InvalidProvider extends Ucanto.Failure {
name: 'InvalidProvider'
}

// Customer
export type CustomerGet = InferInvokedCapability<typeof CustomerCaps.get>
export interface CustomerGetSuccess {
did: AccountDID
}
export interface CustomerNotFound extends Ucanto.Failure {
name: 'CustomerNotFound'
}
export type CustomerGetFailure = CustomerNotFound | Ucanto.Failure

// Consumer
export type ConsumerHas = InferInvokedCapability<typeof ConsumerCaps.has>
export type ConsumerHasSuccess = boolean
export type ConsumerHasFailure = Ucanto.Failure
export type ConsumerGet = InferInvokedCapability<typeof ConsumerCaps.get>
export interface ConsumerGetSuccess {
did: DIDKey
allocated: number
total: number
subscription: string
}
export interface ConsumerNotFound extends Ucanto.Failure {
name: 'ConsumerNotFound'
}
export type ConsumerGetFailure = ConsumerNotFound | Ucanto.Failure

// Subscription
export type SubscriptionGet = InferInvokedCapability<
typeof SubscriptionCaps.get
>
export interface SubscriptionGetSuccess {
customer: AccountDID
consumer: DIDKey
}
export interface SubscriptionNotFound extends Ucanto.Failure {
name: 'SubscriptionNotFound'
}
export type SubscriptionGetFailure =
| SubscriptionNotFound
| UnknownProvider
| Ucanto.Failure

// Rate Limit
export type RateLimitAdd = InferInvokedCapability<typeof RateLimitCaps.add>
export interface RateLimitAddSuccess {
id: string
}
export type RateLimitAddFailure = Ucanto.Failure

export type RateLimitRemove = InferInvokedCapability<
typeof RateLimitCaps.remove
>
export type RateLimitRemoveSuccess = Unit

export interface RateLimitsNotFound extends Ucanto.Failure {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: RateLimits is plural here but singular in all other types

name: 'RateLimitsNotFound'
}
export type RateLimitRemoveFailure = RateLimitsNotFound | Ucanto.Failure

export type RateLimitList = InferInvokedCapability<typeof RateLimitCaps.list>
export interface RateLimitSubject {
id: string
rate: number
}
export interface RateLimitListSuccess {
limits: RateLimitSubject[]
}
export type RateLimitListFailure = Ucanto.Failure

// Space
export type Space = InferInvokedCapability<typeof space>
export type SpaceInfo = InferInvokedCapability<typeof info>
Expand Down Expand Up @@ -147,5 +228,12 @@ export type AbilitiesArray = [
AccessSession['can'],
AggregateOffer['can'],
AggregateGet['can'],
OfferArrange['can']
OfferArrange['can'],
CustomerGet['can'],
ConsumerHas['can'],
ConsumerGet['can'],
SubscriptionGet['can'],
RateLimitAdd['can'],
RateLimitRemove['can'],
RateLimitList['can']
]
Loading
Loading