ETag behaviour violates RFC7232 and related issues

[recmo]

Version information:

go-ipfs version: 0.4.9-dev-db43aab
Repo version: 5
System version: amd64/linux
Golang version: go1.8.1

Type: Bug

Severity: Medium

Description:

While working on #3861, I noticed a couple of major and minor issues with the current gateway ETag behaviour:

1. The ETag is incorrectly formatted: ETags need to be enclosed in double quotes. (RFC7232)
2. The ETag for a request like /ipfs/QmYwAPJzv5CZsnA625s3Xf2nemtYgPpHdWEz79ojWnPbdG/readme is readme, not the hash of the content as one would like.
3. If a If-None-Match header is provided for an /ipns/ URL, it can incorrectly return a 304 Not Modified.
4. /ipns/ could use an ETag to improve performance (see /ipns/ should redirect to /ipfs/ and/or cache #3861).

Point 2 will break configurations where a reverse proxy rewrites requests by adding an /ipfs/hash/ prefix. If the operator changes hash, the request will incorrectly return 304.

Proposal

1. Always use the fully resolved CID from api.ResolveNode as the ETag (also for IPNS).
2. Format the result as a valid "Strong Validator" (double quotes around the encoded CID).

The reason to use the fully resolved CID instead of the IPFS dir root is to better support the above mentioned reverse-proxy-rewrite scenario. A single file update will only result in the ETag for this file to be changed.

Impact

This proposal will change all the ETags. This will briefly result in increased cache-misses until the new ETags are used. After that, the number of cache hits will be strictly higher.

[ghost]

Proposal

1. Always use the fully resolved CID from api.ResolveNode as the ETag (also for IPNS).
2. Format the result as a valid "Strong Validator" (double quotes around the encoded CID).

Thanks, and agreed! 👍 Do you want to give it a try?

[Kubuxu]

👍 for that

Same as @lgierth are you willing to handle it?

A single file update will only result in the ETag for this file to be changed.

This is very good idea. Go for it.

[recmo]

I wrote the code before I wrote the issue. I'll format it into a PR now.

[Kubuxu]

@recmo if something is not done from this issues, please extract it into separate one.