Skip to content

Commit

Permalink
chore: update interop tests
Browse files Browse the repository at this point in the history
  • Loading branch information
achingbrain committed Jan 8, 2024
1 parent fbd5a57 commit 977a354
Show file tree
Hide file tree
Showing 23 changed files with 171 additions and 163 deletions.
42 changes: 30 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,24 +159,42 @@ graph TD;

# 🏭 Code Structure

Helia embraces a modular approach and encourages users to bring their own implementations of interfacing libraries to suit their needs. Helia also ships supplemental libraries and tools including:
Helia embraces a modular approach and encourages users to bring their own implementations of various APIs to suit their needs.

- [`@helia/UnixFS`](./packages/unixfs)
- [`@helia/mfs`](./packages/mfs)
- [`@helia/ipns`](./packages/ipns)
- [`@helia/car`](./packages/car)
- [`@helia/strings`](./packages/strings)
- [`@helia/json`](./packages/json)
- [`@helia/dag-json`](./packages/dag-json)
- [`@helia/dag-cbor`](./packages/dag-cbor)
The basic Helia API is defined in:

These libraries are by no means the "one true implementation", but instead instead provide optionality depending on one's needs.

This repo itself is made up of these packages:
- [`/packages/interface`](./packages/interface) The Helia API

The API is implemented by:

- [`/packages/helia`](./packages/helia) An implementation of the Helia API

Helia also ships a number of supplemental libraries and tools.

These libraries are not intended to be the "one true implementation" of any given API, but are made available for users to include depending on the need of their particular application:

- [./packages/unixfs](./packages/unixfs) The `@helia/unixfs` module
- [./packages/mfs](./packages/mfs) The `@helia/mfs` module
- [./packages/ipns](./packages/ipns) `@helia/ipns`
- [./packages/car](./packages/car) `@helia/car`
- [./packages/strings](./packages/strings) `@helia/strings`
- [./packages/json](./packages/json) `@helia/json`
- [./packages/dag-json](./packages/dag-json) `@helia/dag-json`
- [./packages/dag-cbor](./packages/dag-cbor) `@helia/dag-cbor`

An interface suite ensures everything is compatible:

- [`/packages/interop`](./packages/interop) Interop tests for Helia

## Other modules

There are several other modules available outside this repo:

- [`@helia/delegated-routing-v1-http-api`](https://github.com/ipfs/helia-delegated-routing-v1-http-api) An implementation of the [Delegated Routing v1 HTTP API](https://specs.ipfs.tech/routing/http-routing-v1/) including a server and a client
- [Helia WNFS](https://github.com/shovelers/helia-wnfs) a [WNFS](https://guide.fission.codes/developers/webnative/file-system-wnfs) implementation built on top of Helia
- [`@helia/remote-pinning`](https://github.com/ipfs/helia-remote-pinning) A Helia client for communicating with [IPFS Pinning Services](https://ipfs.github.io/pinning-services-api-spec/)
- [`@helia/http-gateway`](https://github.com/ipfs/helia-http-gateway) An implentation of the [IPFS HTTP Gateway API](https://docs.ipfs.tech/concepts/ipfs-gateway/#gateway-types) built with Helia

# 📣 Project status

Helia v1 shipped in 202303 (see [releases](https://github.com/ipfs/helia/releases)), and development keeps on trucking as we work on initiatives in the [roadmap](#roadmap) and make performance improvements and bug fixes along the way.
Expand Down
4 changes: 0 additions & 4 deletions packages/helia/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -46,10 +46,6 @@
".": {
"types": "./dist/src/index.d.ts",
"import": "./dist/src/index.js"
},
"./hashers": {
"types": "./dist/src/utils/default-hashers.d.ts",
"import": "./dist/src/utils/default-hashers.js"
}
},
"eslintConfig": {
Expand Down
4 changes: 4 additions & 0 deletions packages/helia/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
import { MemoryBlockstore } from 'blockstore-core'
import { MemoryDatastore } from 'datastore-core'
import { HeliaImpl } from './helia.js'
import { libp2pDefaults } from './utils/libp2p-defaults.js'
import { createLibp2p } from './utils/libp2p.js'
import type { DefaultLibp2pServices } from './utils/libp2p-defaults.js'
import type { Helia } from '@helia/interface'
Expand All @@ -40,6 +41,9 @@ export * from '@helia/interface'
export * from '@helia/interface/blocks'
export * from '@helia/interface/pins'

export type { DefaultLibp2pServices }
export { libp2pDefaults }

/**
* DAGWalkers take a block and yield CIDs encoded in that block
*/
Expand Down
2 changes: 1 addition & 1 deletion packages/helia/src/utils/libp2p-defaults.browser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ export interface DefaultLibp2pServices extends Record<string, unknown> {
pubsub: PubSub
}

export function libp2pDefaults (options: Libp2pDefaultsOptions): Libp2pOptions<DefaultLibp2pServices> {
export function libp2pDefaults (options: Libp2pDefaultsOptions = {}): Libp2pOptions<DefaultLibp2pServices> {
return {
peerId: options.peerId,
addresses: {
Expand Down
2 changes: 1 addition & 1 deletion packages/helia/src/utils/libp2p-defaults.ts
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ export interface DefaultLibp2pServices extends Record<string, unknown> {
upnp: unknown
}

export function libp2pDefaults (options: Libp2pDefaultsOptions): Libp2pOptions<DefaultLibp2pServices> {
export function libp2pDefaults (options: Libp2pDefaultsOptions = {}): Libp2pOptions<DefaultLibp2pServices> {
return {
peerId: options.peerId,
addresses: {
Expand Down
30 changes: 22 additions & 8 deletions packages/interop/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -53,27 +53,41 @@
"test:electron-main": "aegir test -t electron-main"
},
"devDependencies": {
"@chainsafe/libp2p-noise": "^14.0.0",
"@chainsafe/libp2p-yamux": "^6.0.1",
"@helia/block-brokers": "^0.0.0",
"@helia/car": "^2.0.0",
"@helia/dag-cbor": "^2.0.0",
"@helia/dag-json": "^2.0.0",
"@helia/interface": "^3.0.0",
"@libp2p/identify": "^1.0.1",
"@helia/ipns": "^4.0.0",
"@helia/json": "^2.0.0",
"@helia/mfs": "^2.0.0",
"@helia/strings": "^2.0.0",
"@helia/unixfs": "^2.0.0",
"@ipld/car": "^5.2.5",
"@ipld/dag-cbor": "^9.0.7",
"@libp2p/interface": "^1.1.1",
"@libp2p/kad-dht": "^12.0.2",
"@libp2p/keychain": "^4.0.5",
"@libp2p/peer-id": "^4.0.3",
"@libp2p/tcp": "^9.0.1",
"@libp2p/peer-id-factory": "^4.0.3",
"@libp2p/websockets": "^8.0.1",
"@multiformats/sha3": "^3.0.0",
"aegir": "^42.0.0",
"blockstore-core": "^4.0.0",
"datastore-core": "^9.0.0",
"helia": "^3.0.0",
"ipfs-core-types": "^0.14.1",
"ipfs-unixfs-importer": "^15.2.3",
"ipfsd-ctl": "^13.0.0",
"it-all": "^3.0.4",
"it-drain": "^3.0.5",
"it-last": "^3.0.4",
"it-map": "^3.0.5",
"it-to-buffer": "^4.0.1",
"kubo": "^0.25.0",
"kubo-rpc-client": "^3.0.1",
"libp2p": "^1.0.1",
"multiformats": "^13.0.0"
"multiformats": "^13.0.0",
"p-defer": "^4.0.0",
"uint8arrays": "^5.0.1",
"wherearewe": "^2.0.1"
},
"browser": {
"./dist/test/fixtures/create-helia.js": "./dist/test/fixtures/create-helia.browser.js",
Expand Down
2 changes: 1 addition & 1 deletion packages/interop/test/car.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import type { Helia } from '@helia/interface'
import type { FileCandidate } from 'ipfs-unixfs-importer'
import type { Controller } from 'ipfsd-ctl'

describe('car interop', () => {
describe('@helia/car', () => {
let helia: Helia
let c: Car
let u: UnixFS
Expand Down
2 changes: 1 addition & 1 deletion packages/interop/test/dag-cbor.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import type { Helia } from '@helia/interface'
import type { PutOptions as KuboAddOptions } from 'ipfs-core-types/src/block/index.js'
import type { Controller } from 'ipfsd-ctl'

describe('dag-json interop', () => {
describe('@helia/dag-cbor', () => {
let helia: Helia
let d: DAGCBOR
let kubo: Controller
Expand Down
2 changes: 1 addition & 1 deletion packages/interop/test/dag-json.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ import type { Helia } from '@helia/interface'
import type { PutOptions as KuboAddOptions } from 'ipfs-core-types/src/block/index.js'
import type { Controller } from 'ipfsd-ctl'

describe('dag-json interop', () => {
describe('@helia/dag-json', () => {
let helia: Helia
let d: DAGJSON
let kubo: Controller
Expand Down
89 changes: 50 additions & 39 deletions packages/interop/test/fixtures/create-helia.browser.ts
Original file line number Diff line number Diff line change
@@ -1,51 +1,62 @@
import { noise } from '@chainsafe/libp2p-noise'
import { yamux } from '@chainsafe/libp2p-yamux'
import { bitswap } from '@helia/block-brokers'
import { identify } from '@libp2p/identify'
import { ipnsValidator, ipnsSelector } from '@helia/ipns'
import { kadDHT, removePublicAddressesMapper } from '@libp2p/kad-dht'
import { webSockets } from '@libp2p/websockets'
import { all } from '@libp2p/websockets/filters'
import { MemoryBlockstore } from 'blockstore-core'
import { MemoryDatastore } from 'datastore-core'
import { createHelia, type HeliaInit } from 'helia'
import { createLibp2p } from 'libp2p'
import { sha3512 } from '@multiformats/sha3'
import { createHelia, libp2pDefaults } from 'helia'
import type { Helia } from '@helia/interface'
import type { Libp2p } from '@libp2p/interface'
import type { DefaultLibp2pServices } from 'helia'

export async function createHeliaNode (init?: Partial<HeliaInit>): Promise<Helia> {
const blockstore = new MemoryBlockstore()
const datastore = new MemoryDatastore()
export async function createHeliaNode (): Promise<Helia<Libp2p<DefaultLibp2pServices>>> {
const defaults = libp2pDefaults()

// dial-only in the browser until webrtc browser-to-browser arrives
const libp2p = await createLibp2p({
transports: [
webSockets({
filter: all
})
],
connectionEncryption: [
noise()
],
streamMuxers: [
yamux()
],
datastore,
services: {
identify: identify()
},
connectionGater: {
// allow dialing loopback
denyDialMultiaddr: () => false
}
})
// allow dialing insecure WebSockets
defaults.transports?.pop()
defaults.transports = [
...(defaults.transports ?? []),
webSockets({
filter: all
})
]

// allow dialing loopback
defaults.connectionGater = {
denyDialMultiaddr: () => false
}

const helia = await createHelia({
libp2p,
// use LAN DHT
defaults.services = {
...(defaults.services ?? {}),
dht: kadDHT({
validators: {
ipns: ipnsValidator
},
selectors: {
ipns: ipnsSelector
},
// skips waiting for the initial self-query to find peers
allowQueryWithZeroPeers: true,

protocol: '/ipfs/lan/kad/1.0.0',
peerInfoMapper: removePublicAddressesMapper,
clientMode: false
})
}

// remove services that are not used in tests
delete defaults.services.autoNAT
delete defaults.services.dcutr
delete defaults.services.delegatedRouting

return createHelia<Libp2p<DefaultLibp2pServices>>({
blockBrokers: [
bitswap()
],
blockstore,
datastore,
...init
libp2p: defaults,
hashers: [
sha3512
]
})

return helia
}
77 changes: 39 additions & 38 deletions packages/interop/test/fixtures/create-helia.ts
Original file line number Diff line number Diff line change
@@ -1,48 +1,49 @@
import { noise } from '@chainsafe/libp2p-noise'
import { yamux } from '@chainsafe/libp2p-yamux'
import { bitswap } from '@helia/block-brokers'
import { identify } from '@libp2p/identify'
import { tcp } from '@libp2p/tcp'
import { MemoryBlockstore } from 'blockstore-core'
import { MemoryDatastore } from 'datastore-core'
import { createHelia, type HeliaInit } from 'helia'
import { createLibp2p } from 'libp2p'
import { ipnsValidator, ipnsSelector } from '@helia/ipns'
import { kadDHT, removePublicAddressesMapper } from '@libp2p/kad-dht'
import { sha3512 } from '@multiformats/sha3'
import { createHelia, libp2pDefaults } from 'helia'
import type { Helia } from '@helia/interface'
import type { Libp2p } from '@libp2p/interface'
import type { DefaultLibp2pServices } from 'helia'

export async function createHeliaNode (init?: Partial<HeliaInit>): Promise<Helia> {
const blockstore = new MemoryBlockstore()
const datastore = new MemoryDatastore()
export async function createHeliaNode (): Promise<Helia<Libp2p<DefaultLibp2pServices>>> {
const defaults = libp2pDefaults()
defaults.addresses = {
listen: [
'/ip4/0.0.0.0/tcp/0'
]
}
defaults.services = {
...(defaults.services ?? {}),
dht: kadDHT({
validators: {
ipns: ipnsValidator
},
selectors: {
ipns: ipnsSelector
},
// skips waiting for the initial self-query to find peers
allowQueryWithZeroPeers: true,

const libp2p = await createLibp2p({
addresses: {
listen: [
'/ip4/0.0.0.0/tcp/0'
]
},
transports: [
tcp()
],
connectionEncryption: [
noise()
],
streamMuxers: [
yamux()
],
datastore,
services: {
identify: identify()
}
})
protocol: '/ipfs/lan/kad/1.0.0',
peerInfoMapper: removePublicAddressesMapper,
clientMode: false
})
}

// remove services that are not used in tests
delete defaults.services.autoNAT
delete defaults.services.dcutr
delete defaults.services.delegatedRouting

const helia = await createHelia({
libp2p,
return createHelia<Libp2p<DefaultLibp2pServices>>({
blockBrokers: [
bitswap()
],
blockstore,
datastore,
...init
libp2p: defaults,
hashers: [
sha3512
]
})

return helia
}
3 changes: 2 additions & 1 deletion packages/interop/test/fixtures/create-kubo.browser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ export async function createKuboNode (): Promise<Controller> {
]
}
}
}
},
args: ['--enable-pubsub-experiment', '--enable-namesys-pubsub']
})
}
3 changes: 2 additions & 1 deletion packages/interop/test/fixtures/create-kubo.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ export async function createKuboNode (): Promise<Controller> {
]
}
}
}
},
args: ['--enable-pubsub-experiment', '--enable-namesys-pubsub']
})
}
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ import { createKuboNode } from './fixtures/create-kubo.js'
import type { Helia } from '@helia/interface'
import type { Controller } from 'ipfsd-ctl'

describe('blockstore', () => {
describe('helia - blockstore', () => {
let helia: Helia
let kubo: Controller

Expand Down
Loading

0 comments on commit 977a354

Please sign in to comment.