feat(server): ⚡️ Subdomain Gateway Using Fastify #31

whizzzkid · 2023-10-25T10:53:32Z

Closes: #16

In this PR:

express -> fastify
implemented route redirection host:port/ns/address/path -> address.ns.host:port/path (subdomain)
implemented subdomain handling and content fetching.
fastify-metrics provides insights into node resources.
anecdotal better performance.

Demo:

helia-fastify.mov

Signed-off-by: Nishant Arora <1895906+whizzzkid@users.noreply.github.com>

SgtPooki

blockers:

e2e tests not running locally or in CI: Error: Timed out waiting 60000ms from config.webServer.
Logging output is not human-readable?

nits:

I prefix on types?

SgtPooki · 2023-10-25T16:46:20Z

src/heliaServer.ts

+      const { address, namespace } = this.parseHostParts(request.hostname)
+      const { url: relativePath } = request
+      this.log('Fetching from Helia:', { address, namespace, relativePath })
+      // raw response is needed to respond with the correct content type.
+      for await (const chunk of await this.heliaFetch.fetch({ address, namespace, relativePath })) {
+        if (type === undefined) {
+          type = await parseContentType({ bytes: chunk, path: relativePath })
+          // this needs to happen first.
+          reply.raw.writeHead(200, {
+            'Content-Type': type ?? DEFAULT_MIME_TYPE,
+            'Cache-Control': 'public, max-age=31536000, immutable'
+          })
+        }
+        reply.raw.write(Buffer.from(chunk))
      }
-      this.log('Requesting content from helia:', request.path)
-      await this.fetchFromHeliaAndWriteToResponse({ response, request })
+      reply.raw.end()


this is awesome but it would be ideal to put the header writing into heliaFetch no? heliaFetch should ideally return an HttpResponse object?

can be a TODO

I was thinking from the POV of moving HeliaFetch as a standalone item, it does not parseConetent (yet!) hence only returns the buffer as an AsyncIterable, leaving parsing to whoever is fetching content using helia.

SgtPooki · 2023-10-25T16:51:31Z

src/index.ts

-// Add the routes
-app.get('/', (req, res) => {
-  res.send('Helia Docker, to fetch a page, call `/ipns/<path>` or `/ipfs/<cid>`')
+const app = Fastify({ logger })


seems like logging isn't as human-readable as before? can we clean that up?

here is some output from startup:

╰─ ✘ 1 ❯ PORT=8888 npm run start:dev > helia-http-gateway@0.0.1 start:dev > npm run build && node dist/src/index.js > helia-http-gateway@0.0.1 build > aegir build --bundle false [09:49:39] tsc [started] [09:49:40] tsc [completed] Helia Started! {"level":30,"time":1698252581247,"pid":57765,"hostname":"Russells-MacBook-Pro-314.local","msg":"Server listening at http://0.0.0.0:8888"}

SgtPooki · 2023-10-25T16:58:43Z

src/heliaFetch.ts

@@ -15,6 +15,12 @@ const ROOT_FILE_PATTERNS = [

 const DELEGATED_ROUTING_API = 'https://node3.delegate.ipfs.io/api/v0/name/resolve/'

+interface IHeliaPathParts {


Suggested change

interface IHeliaPathParts {

interface UrlPathParts {

SgtPooki · 2023-10-25T16:59:38Z

src/heliaFetch.ts

@@ -73,7 +79,7 @@ export class HeliaFetch {
  /**
   * Parse a path into its namespace, address, and relative path
   */
-  public parsePath (path: string): { namespace: string, address: string, relativePath: string } {
+  public parsePath (path: string): IHeliaPathParts {


btw, if we cache the regex in this function as a static (or global), we should see some significant improvements instead of reconstructing this regex on every path

you feel that'll help? I'll change it.

Signed-off-by: Nishant Arora <1895906+whizzzkid@users.noreply.github.com>

…ain-gateway' into feat/fastify-subdomain-gateway * refs/remotes/origin/feat/fastify-subdomain-gateway: fix: e2e running with fastify (#32)

Signed-off-by: Nishant Arora <1895906+whizzzkid@users.noreply.github.com>

* main: (27 commits) feat(e2e): add /api/v0/repo/gc test chore: disable METRICS on CI e2e test runs test: add e2e test for /api/v0/version endpoint feat(server): ⚡️ Subdomain Gateway Using Fastify (#31) feat: use production level docker settings (#26) chore: remove unused playwright init code fix: use active LTS in package.json engines fix: playwright CI node-version=20 test: get clinic flame & doctor output from e2e tests test: e2e updates fix: use HOST constant in healthcheck fix: use HOST constant test: cleanup playwright test code test: add playwright tests feat: move HOST,PORT to src/constants.ts fix: ✏️ Fixing urls. (#23) fix: ✏️ helia-docker -> helia-http-gateway (#22) build(deps): Bump @babel/traverse and depcheck (#13) feat: add health-check (#21) fix(server): 🩹 Using sessionID as a fallback to requests where referer is missing. (#20) ... Signed-off-by: Nishant Arora <1895906+whizzzkid@users.noreply.github.com>

whizzzkid added 2 commits October 25, 2023 04:34

feat(server): ⚡️ Subdomain Gateway Using Fastify

3ab13c3

Signed-off-by: Nishant Arora <1895906+whizzzkid@users.noreply.github.com>

feat(server): 📦️ Add prom client

ca7e1d1

Signed-off-by: Nishant Arora <1895906+whizzzkid@users.noreply.github.com>

whizzzkid requested a review from SgtPooki October 25, 2023 10:53

SgtPooki requested changes Oct 25, 2023

View reviewed changes

whizzzkid and others added 7 commits October 25, 2023 11:50

fix: 🗑️ Remove I prefix from Interfaces

d98c9c8

Signed-off-by: Nishant Arora <1895906+whizzzkid@users.noreply.github.com>

fix: e2e running with fastify (#32)

2f51f95

fix(server): 🎨 Pretty Logging for Fastify

6698356

Signed-off-by: Nishant Arora <1895906+whizzzkid@users.noreply.github.com>

Merge remote-tracking branch 'refs/remotes/origin/feat/fastify-subdom…

37c221a

…ain-gateway' into feat/fastify-subdomain-gateway * refs/remotes/origin/feat/fastify-subdomain-gateway: fix: e2e running with fastify (#32)

fix(server): 🔀 Adding version route for next versioning.

48f73ef

Signed-off-by: Nishant Arora <1895906+whizzzkid@users.noreply.github.com>

fix: Not Reconstructing Regexes.

fcb2341

Signed-off-by: Nishant Arora <1895906+whizzzkid@users.noreply.github.com>

fix: test:e2e-x wait-on

c369d30

SgtPooki self-requested a review October 25, 2023 21:24

fix: allow disabling of prometheus metrics

6059372

SgtPooki approved these changes Oct 25, 2023

View reviewed changes

SgtPooki merged commit 26cb74a into main Oct 25, 2023
1 of 2 checks passed

SgtPooki mentioned this pull request Oct 26, 2023

fix: convert CIDv0 to CIDv1 subdomains #38

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(server): ⚡️ Subdomain Gateway Using Fastify #31

feat(server): ⚡️ Subdomain Gateway Using Fastify #31

whizzzkid commented Oct 25, 2023 •

edited

Loading

SgtPooki left a comment

SgtPooki Oct 25, 2023

whizzzkid Oct 25, 2023

SgtPooki Oct 25, 2023

SgtPooki Oct 25, 2023

SgtPooki Oct 25, 2023

whizzzkid Oct 25, 2023

		@@ -15,6 +15,12 @@ const ROOT_FILE_PATTERNS = [

		const DELEGATED_ROUTING_API = 'https://node3.delegate.ipfs.io/api/v0/name/resolve/'

		interface IHeliaPathParts {

feat(server): ⚡️ Subdomain Gateway Using Fastify #31

feat(server): ⚡️ Subdomain Gateway Using Fastify #31

Conversation

whizzzkid commented Oct 25, 2023 • edited Loading

SgtPooki left a comment

Choose a reason for hiding this comment

SgtPooki Oct 25, 2023

Choose a reason for hiding this comment

whizzzkid Oct 25, 2023

Choose a reason for hiding this comment

SgtPooki Oct 25, 2023

Choose a reason for hiding this comment

SgtPooki Oct 25, 2023

Choose a reason for hiding this comment

SgtPooki Oct 25, 2023

Choose a reason for hiding this comment

whizzzkid Oct 25, 2023

Choose a reason for hiding this comment

whizzzkid commented Oct 25, 2023 •

edited

Loading