DAG visualized as D3 tree

[harlantwood]

I've begun work on the tree visualization discussed in ipfs/ipfs-webui#56

Here is a screenshot of the work in progress:

The source code is here.

Remaining to do:

• Read the hash from the URL instead of hard coding
• Expose api method through gateway: Expose read-only API via Gateway ipfs/kubo#1322

[jbenet]

this looks great. I can't wait! i'll try to get someone to help with ipfs/kubo#1322 sooner

@harlantwood q; does it handle dags well? seems like it would show the same logical node as two different viz nodes (which is fine for exploration, getting the dag to work right would be a huge pain)

[harlantwood]

Great question @jbenet! You're right that as of now it would show a repeated node as two viz nodes. I wonder what we would want it to look like (layout) if we deduplicated nodes... Open to exploring.

The simplest route is probably to have graph shaped visualizations available as well as tree shaped; graphs of course handle this sort of thing well.

Maybe there is a whole body of work on DAG visualization, sort of between tree and graph...

[rht]

+1 for two modes of viz.

Which nomenclature is more natural: unixfs and ipfs view, tree and dag view, (serialized) data and object view, local and web view?

[rht]

http://gateway.ipfs.io/ipfs/QmTkzDwWqPbnAh5YiV5VwcTLnGdwSNsNTn2aDxdXBFca7D/example#/ipfs/QmThrNbvLj7afQZhxH72m5Nn1qiVn3eMKWFYV49Zp2mv9B/graphmd/README.md for some effort deduplication.

[harlantwood]

Which nomenclature is more natural: unixfs and ipfs view, tree and dag view, (serialized) data and object view, local and web view?

I imagine potentially lots of visualizations types, so I imagine just calling them by their viz type: tree, graph, circle packing, treemap, etc... open to other ideas tho.

http://gateway.ipfs.io/ipfs/QmTkzDwWqPbnAh5YiV5VwcTLnGdwSNsNTn2aDxdXBFca7D/example#/ipfs/QmThrNbvLj7afQZhxH72m5Nn1qiVn3eMKWFYV49Zp2mv9B/graphmd/README.md for some effort deduplication.

Thanks! Yup, I based the viz infrastructure on the example viewer infranstructure, and under the hood we're using the refs API like graphmd does.

[harlantwood]

We're now reading the IPFS hash from the URL. To try it out, start up a permissive server:

ipfs daemon --unrestricted-api

Then

git clone git@github.com:ipfs/dataviz.git
cd dataviz
git checkout tree-ltr
cd webapps/tree-ltr
make

This should load up the tree viz in your browser. Change the IPFS hash at the end of the URL to see any other IPFS tree.

[jbenet]

@harlantwood this is awesome. let's make sure to lobby next week to have the gateway changes we need to ship this for use on the public gateways

[harlantwood]

Sounds great

[travisperson]

I would love to help out changing the gateways to support this. I won't make the sprint this week, so unless anyone at the sprint really wants to do it, I can take a lead on it.

[harlantwood]

Awesome @travisperson!

[jbenet]

@harlantwood btw i just tried this with the geoip root (QmQQ3BUpPjgYiTdhp4H9YWSCtoFXs8t91njhpvXNNLd3yB) and realized that because the refs are retrieved recursively, it wont work well for massive graphs. maybe it should grab the refs for each expanded node + its children, but stop there until a child is expanded.

[harlantwood]

I've thought the same thing. Do we have any kind of API calls currently which can specify recursive, depth=1 ?

[harlantwood]

If not, we could fetch each one.

[harlantwood]

Incidentally, I'm working on a graph viz in http://famo.us which will address the DAG data structure much better.

@jbenet what is a good example DAG hash? One that's not just tree data, but has lots of shared leaf nodes?

[jbenet]

@harlantwood not sure, maybe we could import the bitcoin blockchain (duplicates will be addresses), or the linux kernel source tree (probably not many duplicates) or something

[rht]

e.g. go-ipfs dependency graph, where some libs are reused.

[jbenet]

cc @diasdavid

[daviddias]

re: DAG hash - Can we use the one generated by adding all the openWrt versions?

[harlantwood]

Closed with #5

[harlantwood]

Thanks all for the large DAG ideas. We will test working with these and similar in the new breed of visualizations we are making, ala #4.