Skip to content

Commit

Permalink
docs: add TAR format
Browse files Browse the repository at this point in the history
  • Loading branch information
hacdias committed Jun 30, 2022
1 parent 9fc9a9c commit 8023280
Show file tree
Hide file tree
Showing 2 changed files with 67 additions and 1 deletion.
63 changes: 63 additions & 0 deletions IPIP/0000-gateway-tar-response-format.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
# IPIP 0000: Gateway TAR Response Format

- Start Date: (format: 2022-10-10)
- Related Issues:
- https://github.com/ipfs/specs/pull/288
- https://github.com/ipfs/go-ipfs/pull/9029
- https://github.com/ipfs/go-ipfs/pull/9034

# Summary

Add TAR as a response format for the HTTP Gateway.

# Motivation

Currently, the HTTP Gateway only allows the download of single files, or
CAR archives. However, CAR files are sometimes not necessary and user may
want to download entire directories. An example use case is for the IPFS
Web UI, where users are able to download files or directories.

# Detailed design

The solution is to allow the Gateway to support producing TAR archives
by requesting them using either the `Accept` HTTP header or the `format`
URL query.

## Test fixtures

Existing `curl` and `tar` tools can be used by implementers for testing.

Providing static test vectors has little value here, as different TAR libraries may produce
different byte-to-byte files due to unspecified ordering of files and directories inside.

## Design rationale

The current gateway already supports different response formats via the
`Accept` HTTP header and the `format` URL query. This RFC proposes adding
one more supported format to that list.

### User benefit

Users will be able to directly download UnixFs directories from the gateway. In the Web UI,
for example, we will be able to create a direct link to download the file, instead of using the
API to put the whole file in memory before downloading it, saving resources and avoiding bugs.

CLI users will be able to download a directory with existing tools like `curl` and `tar`.

### Compatibility

This RFC is backwards compatible .

### Security

See below.

### Alternatives

An alternative was considered to also support [Gzipped TAR](https://github.com/ipfs/go-ipfs/pull/9034).
However, there is a concern that that may be a vector for DOS attacks since compression requires
high CPU power.

### Copyright

Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/).
5 changes: 4 additions & 1 deletion http-gateways/PATH_GATEWAY.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,7 @@ For example:

- [application/vnd.ipld.raw](https://www.iana.org/assignments/media-types/application/vnd.ipld.raw) – disables [IPLD codec deserialization](https://ipld.io/docs/codecs/), requests a verifiable raw [block](https://docs.ipfs.io/concepts/glossary/#block) to be returned
- [application/vnd.ipld.car](https://www.iana.org/assignments/media-types/application/vnd.ipld.car) – disables [IPLD codec deserialization](https://ipld.io/docs/codecs/), requests a verifiable [CAR](https://docs.ipfs.io/concepts/glossary/#car) stream to be returned
- [application/x-tar](https://developer.mozilla.org/en-US/docs/Web/HTTP/Basics_of_HTTP/MIME_types/Common_types) – disables [IPLD codec deserialization](https://ipld.io/docs/codecs/), requests a [TAR](https://en.wikipedia.org/wiki/Tar_(computing)) archive to be returned
<!-- TODO: https://github.com/ipfs/go-ipfs/issues/8823
- application/vnd.ipld.dag-json OR application/json – requests IPLD Data Model representation serialized into [DAG-JSON format](https://ipld.io/docs/codecs/known/dag-json/)
- application/vnd.ipld.dag-cbor OR application/cbor - requests IPLD Data Model representation serialized into [DAG-CBOR format](https://ipld.io/docs/codecs/known/dag-cbor/)
Expand Down Expand Up @@ -241,7 +242,9 @@ Optional, `format=<format>` can be used to request specific response format.

This is a URL-friendly alternative to sending
`Accept: application/vnd.ipld.<format>` header, see [`Accept`](#accept-request-header)
for more details.
for more details.

In case of `Accept: application/x-tar`, the `?format=` equivalent is `tar`.

<!-- TODO Planned: https://github.com/ipfs/go-ipfs/issues/8769
- `selector=<cid>` can be used for passing a CID with [IPLD selector](https://ipld.io/specs/selectors)
Expand Down

0 comments on commit 8023280

Please sign in to comment.