Run sharness tests from Kubo with PROXY_GATEWAY_URL="http://127.0.0.1:8080"

[lidel]

Until we have coverage and confidence in https://github.com/ipfs/gateway-conformance, the *gateway*.sh sharness tests in https://github.com/ipfs/kubo/tree/master/test/sharness are the only E2E regression and correctness tests we have.

Proposal: let's run them in this repo

We now have PROXY_GATEWAY_URL which enables us to run bifrost-gateway against any HTTP backend that supports GET /ipfs/cid?format=raw requests.

This means we could run bifrost-gateway against regular Kubo gateway and use it as the remote blockstore (and in the future, fetch graphs as CARs via ?format=car).

This would add basic end-to-end tests that catch problems in glue code that is not in go-libipfs, and also allow us to test graph requests in both Kubo and bifrost-gateway, when the time comes.

TODO

• deterministic CAR fixtures test(gateway): use deterministic CAR fixtures ipfs/kubo#9657 to remove surface for bugs caused by change in the way fixtures are generated by a specific Kubo version (now, we only care about static, immutable blocks)
• Github action that checkouts Kubo master or specific commit and runs kubo/tests/sharness/*gateway*.sh against GWAY_PORT of bifrost-gateway
  • prior art: go-libipfs/gateway-sharness.yml
• Kubo's ipfs daemon should run on unique port, and endpoint should be exported as PROXY_GATEWAY_URL
• (tbd) Surgical tweak of kubo/tests/sharness/lib logic to override GWAY_PORT with value from PROXY_GATEWAY_URL if that env is present.
  • This means requests will hit bifrost-gateway that is in front of Kubo, but all the fixture and assert preparation based on kubo CLI like ipfs dag import or ipfs block get will still work, because these do not use GWAY_PORT

[lidel]

We've decided this is not feasible due to reasons listed in #59

It was worth giving it a try in case it "just works", but we are not that lucky, additional work is needed.
Since we want to move away from sharness anyways,

it is way better to invest time into modern test suite tracked in #63