Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move GATEWAY to env and expose ipfs gateway #992

Merged
merged 4 commits into from
Feb 25, 2022
Merged

Move GATEWAY to env and expose ipfs gateway #992

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
045d026
Move GATEWAY to env and expose ipfs gateway
flea89 Feb 16, 2022
b1b99a7
Update miniflare env tpl and remove comment
flea89 Feb 16, 2022
b22ac5c
chore: rename GATEWAY_URL
flea89 Feb 18, 2022
ae18f48
chore: add GATEWAY_URL to envs
flea89 Feb 25, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions packages/api/.env.local.tpl
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,4 +30,7 @@ CLUSTER_API_URL=http://127.0.0.1:9094
# no need for localtunnel, miniflare runs on localhost
PG_REST_URL=http://127.0.0.1:3000

# IPFS Gateaway URL pointing to dockerised ipfs instance
GATEWAY_URL=http://localhost:8080

ENV=dev
6 changes: 3 additions & 3 deletions packages/api/src/car.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import * as cbor from '@ipld/dag-cbor'
import * as pb from '@ipld/dag-pb'
import retry from 'p-retry'
import { InvalidCarError } from './errors.js'
import { GATEWAY, LOCAL_ADD_THRESHOLD, MAX_BLOCK_SIZE } from './constants.js'
import { LOCAL_ADD_THRESHOLD, MAX_BLOCK_SIZE } from './constants.js'
import { JSONResponse } from './utils/json-response.js'
import { getPins, PIN_OK_STATUS, waitAndUpdateOkPins } from './utils/pin.js'
import { normalizeCid } from './utils/cid.js'
Expand Down Expand Up @@ -64,8 +64,8 @@ export async function carGet (request, env, ctx) {
} = request
// gateway does not support `carversion` yet.
// using it now means we can skip the cache if it is supported in the future
const url = new URL(`/api/v0/dag/export?arg=${cid}&carversion=1`, GATEWAY)
res = await fetch(url, { method: 'POST' })
const url = new URL(`/api/v0/dag/export?arg=${cid}&carversion=1`, env.GATEWAY_URL)
res = await fetch(url.toString(), { method: 'POST' })
flea89 marked this conversation as resolved.
Show resolved Hide resolved
if (!res.ok) {
// bail early. dont cache errors.
return res
Expand Down
1 change: 0 additions & 1 deletion packages/api/src/constants.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
export const GATEWAY = new URL('https://ipfs.io')
export const JWT_ISSUER = 'web3-storage'
export const METRICS_CACHE_MAX_AGE = 10 * 60 // in seconds (10 minutes)
export const LOCAL_ADD_THRESHOLD = 1024 * 1024 * 2.5
Expand Down
1 change: 1 addition & 0 deletions packages/api/src/env.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ import pkg from '../package.json'
* @property {string} [CLUSTER_BASIC_AUTH_TOKEN]
* @property {string} PG_REST_URL
* @property {string} PG_REST_JWT
* @property {string} GATEWAY_URL
* @property {string} [S3_BUCKET_ENDPOINT]
* @property {string} [S3_BUCKET_NAME]
* @property {string} [S3_BUCKET_REGION]
Expand Down
4 changes: 1 addition & 3 deletions packages/api/test/cors.spec.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,7 @@ import { endpoint } from './scripts/constants.js'
const cid = 'bafkqaaa'

describe('CORS', () => {
// FIXME: TypeError: terminated for request to ipfs.io gateway...
// I think this is a bug in Miniflare...
it.skip('sets CORS headers', async () => {
it('sets CORS headers', async () => {
const res = await fetch(new URL(`car/${cid}`, endpoint))
assert(res.ok)
assert.strictEqual(res.headers.get('Access-Control-Allow-Origin'), '*')
Expand Down
1 change: 1 addition & 0 deletions packages/api/test/scripts/worker-globals.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ export const S3_SECRET_ACCESS_KEY_ID = 'secret-access-key'
export const DATABASE = 'postgres'
export const PG_REST_URL = 'http://localhost:3000'
export const PG_REST_JWT = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJyb2xlIjoicG9zdGdyZXMifQ.oM0SXF31Vs1nfwCaDxjlczE237KcNKhTpKEYxMX-jEU'
export const GATEWAY_URL = 'http://localhost:8080'

// Can be removed once we get a test mode for admin magic sdk.
export const DANGEROUSLY_BYPASS_MAGIC_AUTH = true
2 changes: 1 addition & 1 deletion packages/api/wrangler.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ bindings = [{ name = "NAME_ROOM", class_name = "NameRoom0" }]
account_id = "fffa4b4363a7e5250af8357087263b3a" # Protocol Labs CF account
zone_id = "7eee3323c1b35b6650568604c65f441e" # web3.storage zone
route = "https://api.web3.storage/*"
vars = { CLUSTER_API_URL = "https://web3.storage.ipfscluster.io/api/", ENV = "production", PG_REST_URL = "https://web3-storage-pgrest-prod.herokuapp.com" }
vars = { CLUSTER_API_URL = "https://web3.storage.ipfscluster.io/api/", ENV = "production", PG_REST_URL = "https://web3-storage-pgrest-prod.herokuapp.com", GATEWAY_URL = "https://ipfs.io" }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add this to staging and other envs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!
We definitively need staging to point to https://ipfs.io

For other cloudfare dev envs I would suggest we keep the https://ipfs.io (rather than the local version) to avoid having to create another tunnel, would you agree?

Honestly, I expect now that we have miniflare, that devs would use that for development rather than this (in most cases) so it shouldn't be a big deal, would you agree?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, let's point to ipfs.io

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cool, ✅


[env.production.durable_objects]
bindings = [{ name = "NAME_ROOM", class_name = "NameRoom0" }]
Expand Down
4 changes: 2 additions & 2 deletions packages/tools/docker/cluster/docker-compose.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,10 +40,10 @@ services:
image: ipfs/go-ipfs:v0.10.0 # update this when go-ipfs M1 macs https://github.com/ipfs/go-ipfs/issues/8645
volumes:
- ipfs0:/data/ipfs
#ports:
ports:
- "8080:8080" # ipfs gateway - expose if needed/wanted
#- "4001:4001" # ipfs swarm - expose if needed/wanted
#- "5001:5001" # ipfs api - expose if needed/wanted
#- "8080:8080" # ipfs gateway - expose if needed/wanted

cluster0:
container_name: cluster0
Expand Down