dag API and resolving paths

[vmx]

Version information:

go-ipfs version: 0.4.14-dev-25eeb1e25
Repo version: 6
System version: amd64/linux
Golang version: go1.9.2

Type: Enhancement, Bug

Description:

I'm currently working on implementing the dag API for https://github.com/ipfs/js-ipfs-api/. There I stumbled upon issues when running the https://github.com/ipfs/interface-ipfs-core/ dag tests. Resolving doesn't work as I expect it, though I might have a wrong understanding. Here's an example:

$ curl -F 'data={"data": "aGVsbG8gd29ybGQh"}' 'http://localhost:5001/api/v0/dag/put?hash=sha2-256&format=dag-pb&input-e
{"Cid":{"/":"QmTy3X89nStNtmyth2iEtEC6VFchCpZUGtsJWYNTux6hVz"}}
$ curl 'http://127.0.0.1:5001/api/v0/dag/get?arg=QmTy3X89nStNtmyth2iEtEC6VFchCpZUGtsJWYNTux6hVz'
{"data":"aGVsbG8gd29ybGQh","links":[]}
$ curl 'http://127.0.0.1:5001/api/v0/dag/resolve?arg=QmTy3X89nStNtmyth2iEtEC6VFchCpZUGtsJWYNTux6hVz/Data'
{"Message":"no link by that name","Code":0,"Type":"error"}

I would expect the last curl command to return:

"aGVsbG8gd29ybGQh"

Am I wrong?

[vmx]

This is based on a test interface-ipfs-core: in https://github.com/ipfs/interface-ipfs-core/blob/0c76cddcc7eaf8915ca11b20d6e41c7271b57bea/js/src/dag.js#L242-L247.

[vmx]

CBOR works as I'd expect it:

$ curl -F 'data={"hello":"world!"}' 'http://localhost:5001/api/v0/dag/put?hash=sha2-256&format=dag-cbor&input-enc=json'
{"Cid":{"/":"zdpuAvk4Um53YugGaXLJWLjmMnNGyX4JpvtzDS6PK98Sy9XfQ"}}
$ curl 'http://127.0.0.1:5001/api/v0/dag/get?arg=zdpuAvk4Um53YugGaXLJWLjmMnNGyX4JpvtzDS6PK98Sy9XfQ'
{"hello":"world!"}
$ curl 'http://127.0.0.1:5001/api/v0/dag/resolve?arg=zdpuAvk4Um53YugGaXLJWLjmMnNGyX4JpvtzDS6PK98Sy9XfQ/hello'
{"Cid":{"/":"zdpuAvk4Um53YugGaXLJWLjmMnNGyX4JpvtzDS6PK98Sy9XfQ"},"RemPath":"hello"}

[magik6k]

Yep, that's definitely a bug, it's something that should get resolved once #4666 is addressed.

[vmx]

I had a quick chat with @whyrusleeping on IRC about this. Here's the summary:

In the Go implementation slashes always mean "traverse links". So <cid>/somename means "follow the link with the name "somename". So the data of a ProtoBufs encoded node (here represented as JSON) would look like this:

{
  "Data": <some-base64-data>,
  "Links": [
    {"somename": { "/": <cid> }}
  ]
}

There is no way to get the data of a DAG node through a path.

[magik6k]

There is no way to get the data of a DAG node through a path.

Yes, but the other graph-related functions should still work. Git plugin is one example here.

In the Go implementation slashes always mean "traverse links"

This is the real problem here, it's going to be much simpler to address once /ipld is separated from /ipfs a. See #4666 for more details

[vmx]

@magik6k I've read #4666 before, but know I understand the /ipld - /ipfs separation. Does it mean that the test linked in my comment above #4728 (comment) should work once #4666 is solved?

[magik6k]

It will require some more changes to the internals. Dag api will use ipld resolving by default, so it will work.

[magik6k]

The todos here are:

• Solve Think through the CoreAPI Path API and implementation. #4666
• Rewrite dag commands to use CoreAPI
• Make merkledag nodes implement ipld properly and fix unixfs resolver

[magik6k]

We now have separate /ipfs and /ipld paths.

• Resolving /ipfs works as expected, as it's resolving unixfs paths and falling back to IPLD paths if resolver for a given node type is not available
• /ipld paths still need to be fixed (requires changes in go-merkledag)

[rvagg]

Hey hey, I just stumbled on this from a js-ipfs code comment, I think it's fixed now as of v0.10. I've uncommented the tests in js-ipfs that were blocked waiting for this ipfs/js-ipfs#3917

@aschmahmann I think this can be closed now.

[aschmahmann]

Correct @rvagg we can now do ipfs dag get /ipld/QmPR5kgqNpHpxnAUL22GsjUoBY4y3sJ4V7KrMnqc99C66z/Data, thanks for reporting back 😄. Closed by #7976

I don't think ipfs dag resolve really has a place here, but if someone does they can feel free to reopen or file a new issue.