Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add HTTP Gateway Specs #283

Merged
merged 26 commits into from
Jul 1, 2022
Merged
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
a28d1de
feat: initial HTTP gateway specs
lidel Jun 2, 2022
6e24eb0
gateway: add Content-Range
lidel Jun 3, 2022
2e4374f
gateway: registerProtocolHandler uri router
lidel Jun 3, 2022
101fa5e
CODEOWNERS: add lidel for ./http-gateways
lidel Jun 3, 2022
7be0611
gateway: resolving an advanced DNSLink chain
lidel Jun 8, 2022
6a0e2fc
gateway: only-if-cached HEAD behavior
lidel Jun 8, 2022
13f53a8
gateway: suggestions from reviewers
lidel Jun 8, 2022
a414411
gateway: include CIDv1 node in summary
lidel Jun 8, 2022
af0363e
gateway: reorder URI router section
lidel Jun 8, 2022
4156b43
gateway: add Denylists section
lidel Jun 8, 2022
5435910
gateway: switch only-if-cached miss to 412
lidel Jun 9, 2022
176133a
gateway: apply suggestions from review
lidel Jun 9, 2022
cad7046
gateway: apply suggestions from Cloudflare
lidel Jun 15, 2022
04111e6
gateway: add X-Content-Type-Options
lidel Jun 15, 2022
ce193e6
gateway: simplify dnslink summary
lidel Jun 23, 2022
ffe8af9
gateway: document 412 Precondition Failed
lidel Jun 23, 2022
7e65a76
gateway: link to ipld codecs explainer
lidel Jun 23, 2022
e3637e5
gateway: stub about handling traversal errors
lidel Jun 23, 2022
2c731d3
gateway: expand HTTP caching considerations
lidel Jun 23, 2022
6787fd8
gateway: editorial fixes
lidel Jun 23, 2022
25497a4
gateway: expand on Host header parsing
lidel Jun 23, 2022
4b57971
gateway: editorial fixes
lidel Jun 23, 2022
75543c3
gateway: X-Forwarded-Proto and X-Forwarded-Host
lidel Jun 24, 2022
849a7e3
gateway: editorial fixes
lidel Jun 24, 2022
e06249f
gateway: X-Trace-Id
lidel Jun 24, 2022
9fc9a9c
gateway: Generated HTML with directory index
lidel Jun 28, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Spec Stewards defined below are automatically requested for review when
# someone opens a pull request that modifies area of their interest.

http-gateways/ @lidel
12 changes: 5 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,14 +29,12 @@ The specs contained in this repository are:
- [Protocol Architecture Overview](./ARCHITECTURE.md) - the top-level spec and the stack
- [Other IPFS Overviews](/overviews) - quick overviews of the various parts of IPFS
- **User Interface (aka Public APIs):**
- [Core API (aka using IPFS as a package/module)](./API_CORE.md)
- [JavaScript Interface](https://github.com/ipfs/interface-js-ipfs-core)
- [Golang Interface](https://github.com/ipfs/interface-go-ipfs-core)
- [CLI (the ipfs daemon API)](./API_CLI.md)
- [HTTP API](./API_HTTP.md)
- HTTP Gateway
- [HTTP Gateways](./http-gateways/README.md) - implementation agnostic interfaces for accessing content-addressed data over HTTP
- IPFS implementations may provide additional interfaces, for example:
- [HTTP RPC API exposed by go-ipfs](https://docs.ipfs.io/reference/http/api/)
- [Programmatic Core API for JavaScript](https://github.com/ipfs/js-ipfs/tree/master/docs/core-api#readme)
- **Data Formats:**
- [IPLD](https://github.com/ipld/spec) - InterPlanetary Linked Data.
- [IPLD](https://ipld.io/specs/) - InterPlanetary Linked Data.
- [Merkle DAG (Deprecated)](./MERKLE_DAG.md)
- Self Describing Formats ([multiformats](http://github.com/multiformats/multiformats)):
- [multihash](https://github.com/multiformats/multihash) - self-describing hash digest format.
Expand Down
76 changes: 76 additions & 0 deletions http-gateways/DNSLINK_GATEWAY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# DNSLink Gateway Specification

![](https://img.shields.io/badge/status-wip-orange.svg?style=flat-square)

**Authors**:

- Marcin Rataj ([@lidel](https://github.com/lidel))

----

**Abstract**

DNSLink Gateway is an extension of
[PATH_GATEWAY.md](./PATH_GATEWAY.md)
that enables hosting a specific content path under a specific DNS name.

This document describes the delta between [PATH_GATEWAY.md](./PATH_GATEWAY.md) and this gateway type.

In short:

- HTTP request includes a valid DNSLink name in `Host` header
lidel marked this conversation as resolved.
Show resolved Hide resolved
- gateway resolves DNSLink to an immutable content root identified by a CID
- HTTP response includes the data for the CID
- No third-party CIDs can be loaded
lidel marked this conversation as resolved.
Show resolved Hide resolved

# Table of Contents

- [DNSLink Gateway Specification](#dnslink-gateway-specification)
- [Table of Contents](#table-of-contents)
- [HTTP API](#http-api)
- [`GET /[{path}][?{params}]`](#get-pathparams)
- [`HEAD /[{path}][?{params}]`](#head-pathparams)
- [HTTP Request](#http-request)
- [Request headers](#request-headers)
- [`Host` (request header)](#host-request-header)
- [Appendix: notes for implementers](#appendix-notes-for-implementers)
- [Leveraging DNS for content routing](#leveraging-dns-for-content-routing)

# HTTP API

## `GET /[{path}][?{params}]`

Downloads data at specified path under the content path for DNSLink name provided in `Host` header.

- `path` – optional path to a file or a directory under the content root sent in `Host` HTTP header
- Example: if `Host: example.com` then the content path to resolve is `/ipns/example.com/{path}`

## `HEAD /[{path}][?{params}]`

Same as GET, but does not return any payload.

# HTTP Request

## Request headers

### `Host` (request header)


Defines the DNSLink name to resolve into `/ipfs/{cid}/` prefix that should be
prepended to the `path` before the final IPFS content path resolution is
performed.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we define that:

  • a dnslink that points to a /ipns path must be resolved to an immutable path first
  • a dnslink with a path component should be resolved to it's target CID

...path before appending the request path

Copy link
Member Author

@lidel lidel Jun 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@olizilla I've added this + note about recursion limit + an example in 7be0611 – lmk if that covers all the bases


Example: if client sent HTTP GET request for `/sub-path` path and `Host:
example.com` header, and DNS at `_dnslink.example.com` has TXT record with
value `dnslink=/ipfs/cid1`, then the final content path is
`/ipfs/cid1/sub-path`

# Appendix: notes for implementers
lidel marked this conversation as resolved.
Show resolved Hide resolved

## Leveraging DNS for content routing

- It is a good idea to publish
[DNSAddr](https://github.com/multiformats/multiaddr/blob/master/protocols/DNSADDR.md)
TXT records with known content providers for the data behind a DNSLink. IPFS
clients will be able to detect DNSAddr and preconnect to known content
providers, removing the need for expensive DHT lookup.
Loading