Background on Address Scheme discussions (dweb, ipfs:// and more) #152
Conversation
There was a problem hiding this comment.
Thanks for pulling all this info together.
I think that what's also needed is:
- the current spec, fleshed out.
- a proposed different spec, fleshed out, that achieves the same things (re supporting ipfs, ipns, ipld).
that way, we can compare and see what is more complex, pros/cons, and so on.
it is ok to do "one thing for now" and "one thing for the future". This is often how upgrading happens. We don't have to argue about the second thing with anyone who is hostile to it. if the former thing satisfies them, that should be good enough for them.
addresses-scheme/readme.md
Outdated
| ### Why it's not good enough | ||
|
|
||
| #### Reason 1: We want IPFS, IPNS and IPLD to be handled by a single schema | ||
| Creating an `ipfs:` schema would not be enough because `ipfs:` only refers to mutable content. You would, at the very least, need an `ipns:` schema too. |
There was a problem hiding this comment.
ipld: would be needed too. as important as ipns:
addresses-scheme/readme.md
Outdated
|
|
||
| also | ||
|
|
||
| > A minor reason is not having to force people to swallow N shemes (ipfs:// ipns:// ipld:// and counting), and instead use one that muxes. |
There was a problem hiding this comment.
I now think this is a much bigger reason than that email suggested. Even discounting the "Unify the filesystem-database-web rift" rift, I would still press for this construction for this reason.
There was a problem hiding this comment.
note that it may be more than 3 in the future: /dns, /ens, /btc . NOT doing this now effectively shuts down future possibilities by making them "not worth doing the work of introducing a URL scheme".
There was a problem hiding this comment.
I feel it should be explicitly stated in the document if this single scheme aims to be for IPFS ecosystem only, or something that can be adopted by other distributed systems.
If we want to share it with others, eg. Ethereum, Tor Onion Services etc, then I see potential problem with single scheme: who is responsible for muxing when multiple vendors are in play?
Let's say we have: dweb://ipfs/Qmbar.. and dweb://foo/buz. If IPFS browser add-on provides handler for dweb://, FOO is unable to handle its own URIs.
We should plan for this now, and have contingency written down..
There was a problem hiding this comment.
Which I guess, means that dweb support has to be in its own package, so that tor wouldn't depend on ipfs...
addresses-scheme/readme.md
Outdated
| There are a few goals tugging against each other: | ||
| 1. The Noble Goal: Unify the filesystem-database-web rift | ||
| 2. The Pragmatic Goal: Do the obvious, easy thing. | ||
| 3. The Design Goal: Create Addresses that People will Love Using |
There was a problem hiding this comment.
Setting this document up this way -- calling one thing "the noble thing" and another "the pragmatic thing" -- strongly biases away from the former and towards the second. For many engineers, the framing "noble vs pragmatic" immediately discounts whatever is called noble. Therefore, I strongly disagree with this framing. This is not that hard, it's just confusing now because we haven't presented one doc with it.
addresses-scheme/readme.md
Outdated
| @@ -0,0 +1,72 @@ | |||
| (Unresolved) Address Scheme for IPFS | |||
There was a problem hiding this comment.
"Address Scheme for IPFS" is not unresolved. We've had a clear path for a long time and going away from that is undoing decisions. This is the current address scheme, and is not an "unresolved thing waiting for a decision". There are suggestions to undo that decision and change to something else, for the sake of simplifying implementation of browser bridges, and satisfying people's URL orthodoxy.
Perhaps "(Unresolved) Changing the URL address scheme IPFS uses"
addresses-scheme/readme.md
Outdated
| There are a few goals tugging against each other: | ||
| 1. The Noble Goal: Unify the filesystem-database-web rift | ||
| 2. The Pragmatic Goal: Do the obvious, easy thing. | ||
| 3. The Design Goal: Create Addresses that People will Love Using |
There was a problem hiding this comment.
- "avoid polluting the scheme namespace with multiple schemes" is an important goal to add here, as its own thing.
addresses-scheme/readme.md
Outdated
|
|
||
| # Backround for the Discussions about an Address Scheme for IPFS and `ipfs:` URIs | ||
|
|
||
| The discussions around `ipfs:` vs. `fs:` vs. `dweb:` is a confusing one that's been going on since @jbenet published the [IPFS whitepaper](https://ipfs.io/ipfs/QmR7GSQM93Cx5eAg6a6yRzNde1FQv7uL6X1o4k7zrJa3LX/ipfs.draft3.pdf). |
There was a problem hiding this comment.
worth noting:
fs:is historical. can be gotten rid of (still early enough).dweb:is a recent proposal.
addresses-scheme/readme.md
Outdated
|
|
||
| ## The Design Goal: Create Addresses that People will Love Using | ||
|
|
||
| From a design perspective, the challenge is to create a schema that makes intuitive sense, maximizes possibilities, and allows people to identify content with addresses that are reliable, powerful, and pleasant to use. |
There was a problem hiding this comment.
this is a great point, well phrased.
though this also runs counter to hashes in addresses :) -- hashes are not particularly pleasant to use.
addresses-scheme/readme.md
Outdated
|
|
||
| @jbenet agreed to that pragmatism: | ||
| > **These goals are secondary in time to getting browser adoption. Meaning that we CAN do things like recommend ipfs:// ipns:// ipld://** IF browser vendors think that it's unlikely to get adoption this way now. We can work on unifying the fs-db-web rift later. **We're not dogmatic, we're pragmatic.** But we want to make sure we push in the right places and try to make as much as we can better. | ||
|
|
There was a problem hiding this comment.
also worth adding that some current browsers have trouble with the construction: origin = a:/b/c because they expect: `origin = a:/b, even though that's not in the URI spec.
There was a problem hiding this comment.
this is reason enough to provide a different mapping to the same underlying scheme.
meaning, make:
- I'm ok with allowing
ipfs://<hash>to work, for the sake of current expectations in firefox. - But this is not reason enough to abandon goals. we can just make those goals cause less friction by allowing both.
There was a problem hiding this comment.
Also, worth noting:
- browsers currently assume
<origin>is a case insensitive thing. - so a construction like
dweb://ipfs/<base58-hash>or evenipfs://<base58-hash>doesn't work without redirection or translation in the implementation. - today, given firefox constraints, what is actually fed into firefox as the proper URL is
ipfs://<base16-or-32-hash>(this does not have to be the thing that shows up to the user or that we encourage people to use as links).
|
I felt kind of ignored, so I added a section on my concerns with I know it is long, but I've put time into making real examples with real testable code and concrete examples. |
|
Which I guess, means that `dweb` support has to be in its own package,
so that tor wouldn't depend on ipfs...
Edit: githubs email reply feature replied to the wrong place.
…On 03/04/2017 05:52 PM, Marcin Rataj wrote:
***@***.**** commented on this pull request.
------------------------------------------------------------------------
In addresses-scheme/readme.md
<#152 (comment)>:
> +
+### Unify the filesystem-database-web rift
+
+In conversations documented [here](ipfs/in-web-browsers#4), @jbenet and @Gozala cover this topic relatively concisely.
+
***@***.*** explained his rationale:
+
+> The major reason has to do with unifying FSes, Databases, and the Web with a singular way of addressing all data. It's about undoing the harm that URLs brought unto computing systems by fragmenting the ecosystem. To this day the rift between both worlds prevents simple tooling from working with both, and has much to do with the nasty complexity of working with networked data all the modern target platforms. Sorry, this may sound vague, but it's very specific: addressing of data broke when URLs and URIs were defined as a space OUTSIDE unix/posix paths, instead of INSIDE unix/posix paths (unlike say plan9's 9p transparent addressing). This made sense at the time, but it created a division that to this day force "the web" and "the OS" to be very distinct platforms. Things can be much better. Mobile platforms, for one, have done away with the abstractions in the user facing parts, hiding away the rift from users, and only forcing developers to deal with it (clearly a better UX), but problems still exist, and many apps are hard to write because of it. we'd like to improve things, particularly since "a whole new world" of things is joining the internet (blockchains, ipfs, other decentralized web things). It would be nice if there's a nice compatible way to bridge with the web's expectations (dweb://...) but work towards fixing things more broadly.
+
+also
+
+> we'd like to improve things, particularly since "a whole new world" of things is joining the internet (blockchains, ipfs, other decentralized web things). It would be nice if there's a nice compatible way to bridge with the web's expectations (dweb://...) but work towards fixing things more broadly.
+
+also
+
+> A minor reason is not having to force people to swallow N shemes (ipfs:// ipns:// ipld:// and counting), and instead use one that muxes.
I feel it should be explicitly stated in the document if this single
scheme aims to be for IPFS ecosystem only, or something that can be
adopted by other distributed systems.
If we want to share it with others, eg. Ethereum, Tor Onion Services
etc, then I see potential problem with single scheme: who is
responsible for muxing when multiple vendors are in play?
Let's say we have: |dweb://ipfs/Qmbar..| and |dweb://foo/buz|. If IPFS
browser add-on provides handler for |dweb://|, FOO is unable to handle
its own URIs.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#152 (comment)>, or
mute the thread
<https://github.com/notifications/unsubscribe-auth/ABU7-DaUQp2cR7Q07epkz2FpbaAqXuJZks5riZa7gaJpZM4MS0KU>.
|
flyingzumwalt
force-pushed
the
address-scheme
branch
3 times, most recently
from
78c416c
to
7a77a02
Compare
|
I've added info from everyone's comments, except @timthelion -- provided suggestions on his PR. I also rearranged the document so that (hopefully) it will be easy for people to submit further corrections/additions, etc. Please keep your additions concise with lots of links out to the ongoing discussions in GH issues and other PRs. Note: I'm going to be very busy traveling and presenting at a conference this week, so I will not have much bandwidth to guide this discussion or to keep it moving forward. |
|
Just to let you konw, I will not be commenting further on this issue. |
|
Why @timthelion? You're making good points. I'm trying to gather the info and discussions so that your contributions move this discussion forward, rather than being yet another github issue that lies unresolved. Have I done something to discourage you? |
|
@flyingzumwalt don't worry, you didn't do anything wrong or anything to discourage me. I put several hours into explaining my position, I explained it as well as I could and I don't beleive that I could do so better than I have already. But the real reason why I do not want to continue, is that this discussion is making me feel unhappy, frustrated, and combative. For the sake of my own mental health, I would prefer not to continue with this discussion. I just feel like this sometimes. Perhaps it is the coldness of text. |
|
I can certainly empathize with that @timthelion. 😓 If it's any consolation, the delay on getting a response to your final posix/unix questions was because the people who can give a good answer are busy with other stuff right now, but I'm going to make sure we get good answers on the record eventually. I wanted to make sure they answer your new questions rather than re-answering stuff that's already been discussed elsewhere or re-hashing debates that already happened. In order to do that, I needed to gather everything into a document that provides the context/history. So please don't delete your fork with those comments and examples. If you're willing, it will help me cite your comments and avoid losing them over time if you put your comments into a separate file next to the readme.md -- ie. a file called "timthelion-comments.md" and submit a PR with that file in it. Alternatively, it's perfect material for a blog post. If you put it on your blog we can cite it there and save a copy on ipfs. |
|
And I completely agree about the coldness of text-based communication. It can be really rough on the human heart. |
addresses-scheme/readme.md
Outdated
| ### Criticisms of this Approach | ||
|
|
||
| #### Criticism 1: We want IPFS, IPNS and IPLD to be handled by a single schema | ||
| Creating an `ipfs:` schema would not be enough because `ipfs:` only refers to mutable content. You would, at the very least, need an `ipns:` schema too. |
There was a problem hiding this comment.
`ipfs:` only refers to mutable content
This is a typo, right? It should be:
`ipfs:` only refers to immutable content
There was a problem hiding this comment.
yes that's a typo. you're right. I need to fix that.
|
@jbenet mentions the case insensitivity problem. I think the "Strengths" or "Criticisms" section of each approach should say whether the multihash is in the case insensitive part of the URI. (If an approach requires a browser to hack the URI to not conflate |
|
@jefft0 "URLs should treat upper case letters as equivalent to lower case in scheme names" https://www.ietf.org/rfc/rfc1738.txt So long as http://gateway.ipfs.io/ipfs/Qmaqx89FcFCQ3EocZZEbzGf7mJZpKBe8XAZt8ijoLXkojN works, then why would |
addresses-scheme/readme.md
Outdated
|
|
||
| #### Strength: Fits with the way browsers identify origins | ||
|
|
||
| When enforcing the Single Origin Policy, some current browsers expect: `origin = a:/b`, even though that's not what the URI spec calls for. The `ipfs:` approach fits with that expectation, with URIs like `ipfs:/<root-hash>/foo/bar`, the origin would be `ipfs:/<root-hash>` |
There was a problem hiding this comment.
I'm confused. The spec in the pull request looks like ipfs://ipfs/<root-hash>/foo/bar, not ipfs:/<root-hash>/foo/bar as you wrote here. So the origin would be ipfs://ipfs. Do you want to refer to a different draft of the ipfs: scheme?
There was a problem hiding this comment.
Interesting point.
A couple of things I will have to update the doc to reflect:
- @pawal's spec at First draft of ipfs: URI spec, see #138 #139 includes a brief reference to the
fs:scheme which is fleshed out in @lgierth's https://github.com/ipfs/specs/tree/doc/fs-paths/fs-paths - A couple of points are implicit in @jbenet's arguments, which hang off the fact that an
ipfs:scheme should, due to its name, only return ipfs content:- if you create an
ipfs:scheme, the/ipfs/in the path is redundant. - if you create an
ipfs:scheme you will end up needing more schemes:ipns:,ipld:, etc.
- if you create an
|
@timthelion: Yes, that's what the RFC says about the scheme part of |
ghost
mentioned this pull request
|
Thanks so much @flyingzumwalt, excellent write up ❤️ and thanks for covering for me. I failed at producing any useful writeup during the in-web-browsers sprint. I'll try my best to reframe this from a closer-to-implementation perspective that'll help us quickly wrap up the discussion. I won't repeat the arguments for and against the various ideas proposed here and in older discussions, this PR already does a good job at that. Also @jbenet is right:
We've had lengthy and fruitful discussions spanning years, and consensus about the long term goal of path addressing has been reached long ago (see ipfs/kubo#1678 (comment)). What is up for technical debate is what we usually call the upgrade path -- how to integrate with existing protocols and infrastructure in a way that:
Big protocol migrations happen over decades, and it's fundamentally important that new protocols interoperate well, otherwise the migration cost is simply too high. For example the IPv4/IPv6 migration came up with half a dozen IPv4-over-IPv6 and vice versa sub-protocols. Note that for the upgrade path, we're not constrained to exactly what currently is. Many of the web's protocols (HTML, URLs, etc.) are so-called living standards, which aim to evolve relatively organically. We can propose new APIs and features, weigh in on discussions, and generally participate in the specification and development effort, as we did for example with the Suborigin working group. Browser developers appear to not be generally opposed to the technicalities of dweb: URIs (and sometimes even inviting), if and only if we make an effort to make them play nice with the existing browser platform. While browsers play a big role for ipfs (see ipfs/in-web-browsers) and are mentioned here a lot, we also want dweb: URIs to be supported in all kinds of tools/applications/libraries. One example is @Kubuxu's weechat plugin, which makes the URIs clickable. The four stages of the upgrade path for path addressing.Note that whenever we talk about these, it's important to be very exact in the notation of the address. The number of slashes matters a great deal.
These are not fixed serial steps, instead there's more of a wibbly-wobbly concurrence/simultaneity of all four stages. Networks are a wild mess of parts developing into differing directions, at varying speed (just like a rhizome). We definitely want to make the first three work in the foreseeable future, and also the fourth, although that one admittedly looks like a long shot at the moment :) Regarding NURIs, I'd like to emphasize a comment @btrask wrote,
As a reference, the filesystem ideas of Plan 9 have been the inspiration for NURIs. |
|
Is there a CID spec that shows how to unambiguously distinguish a base32 CID from a base58 CID? (Can't find the CID spec anywhere.) |
Never mind. The multibase code handles it. With Google I found https://github.com/ipld/cid . (It wasn't linked to from https://github.com/ipfs/specs or https://github.com/ipld/specs/blob/master/README.md or https://github.com/ipld/specs/blob/master/ipld/README.md . Maybe it should be?) |
|
I made a related comment in ipfs/in-web-browsers#6 (comment) |
@jefft0 a couple months ago I walked someone through that in detail: https://botbot.me/freenode/ipfs/2017-01-04/?msg=78819903&page=9 I never found the time afterwards to put this in a document however :/ |
ghost
mentioned this pull request
|
No, it would interpret it as an absolute path on the current host. Try this link for example. That was a link to |
|
Yeah, the spec for href specifically expects it to be a URI, so if you want to use this new format you'll likely need a new attribute added to the HTML spec. |
|
It'll work just fine with I'll ping this thread once the DWeb Addressing spec is finally there (give me a week or two more). An outline and introduction have been merged in #167. We're looking for open questions and spots that need clarification. |
This still needs work, but it gives a starting point for people to clarify the discussion.
…cola jbenet and gozala are
daviddias
force-pushed
the
address-scheme
branch
from
7a77a02
to
0e0f8c9
Compare
|
Note to self: use this PR as a place to incorporate existing resources listed at https://github.com/ipfs/in-web-browsers/issues/147 (including ADDRESSING.md / https://docs.ipfs.io/guides/guides/addressing/) |
|
@lidel alive check on this... |
|
Ack, specs consolidation was on the backburner due to other priorities but I hope to tackle it in 2020Q2 at some point (after subdomain gateways land everywhere). I think we should close this outdated PR just to decrease noise and point people at https://github.com/ipfs/in-web-browsers/issues/147 which has a proper timeline of all decisions, making it a much better entry point than this PR. @hsanjuan can you close this and move mentioned issue to ipfs/specs? |
This still needs work, but it gives a starting point for people to clarify the discussion. Please submit comments and/or changes.
ref ipfs/ipfs#227 ipfs/in-web-browsers#4 ipfs/in-web-browsers#28 #139