Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate filesystem merkle-path from IPLD merkle-path #60

Closed
wants to merge 4 commits into from

Conversation

mildred
Copy link
Contributor

@mildred mildred commented Jan 7, 2016

This is an attempt at solving "Accessing link properties problem (important issue to resolve)" from PR #37

TODO items:

I proposed two solutions, and one with a variant, corresponding to these three ideas

Introductions to the changes:

What is a merkle-path?

A merkle-path is a unix-style path (e.g. /a/b/c/d) which initially dereferences through a merkle-link and allows access of elements of the referenced node and other nodes transitively.

There is no single merkle-path, but there are two:

  • merkle-path for filesystems: this is a merkle-path that is designed to be used in the context of filesystems (that also includes network protocols such as HTTP or FTP). Their idea is to be as close as possible to the traditional filesystem semantic
  • merkle-path for IPLD: this is a merkle-path that can be used to access more elements of the IPLD data model (specifically: link properties) but that doesn't fit within the traditional filesystem model.

When you use a merkle path, make sure of which one you use. Command line tools are encouraged to allow switching between the two flavors using a switch.

@jbenet jbenet added the backlog label Jan 7, 2016
@mildred mildred mentioned this pull request Jan 7, 2016
5 tasks
@mildred mildred force-pushed the ipld-spec-linkprop branch from 1eccaeb to 9f3acf2 Compare January 7, 2016 14:08

Path elements are suffixed by either `.link` to access the link properties or by `.object` to dereference the _merkle-link_. if no suffix is present, the _merkle-link_ is dereferenced (to be compatible with _filesystem merkle-paths_ in most cases)

**FIXME**: perhaps use different suffixes so we are less likely to have ambiguities. Using a character that is denied by Windows would be a good idea since those are less likely to be present in filenames. For most cases, this would make _IPLD merkle-paths_ a superset of _filesystem merkle-paths_. For example we could use `?link` and `?object`
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

being able to access link properties includes making it possible to do so via HTTP

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not intend to use the ? character to enable a special HTTP feature, but we could imagine it without problem. That's a separate issue for me.

@mildred mildred force-pushed the ipld-spec-linkprop branch from 8b027ac to c081ae1 Compare January 8, 2016 08:17
@mildred
Copy link
Contributor Author

mildred commented Jan 8, 2016

Commit Talk about escaping keys in merkle-paths is taken from PR #59 and commit Consideration about key escaping is additional remarks for IPLD merkle-paths in case we use escaping.

@mildred
Copy link
Contributor Author

mildred commented Jan 8, 2016

What path syntax do you prefer?

  • fist solution: /ipfs/QmCCC...000/stuff/foo.object/other/cat.link
  • fist solution with another character: /ipfs/QmCCC...000/stuff/foo?object/other/cat?link
  • fist solution with yet another character: /ipfs/QmCCC...000/stuff/foo:object/other/cat:link
  • second solution: /ipfs/QmCCC...000/stuff.foo/other.cat\.jpg
  • second solution with another character: /ipfs/QmCCC...000/stuff|foo/other|cat.jpg
  • second solution and allowing either / or . (or |) if there is no ambiguity

Perhaps more people should express their opinions on those. As for me, I prefer the first solution (that's why I put it first). I generally use prefixes and suffixes when I need to encode things in file names.

@mildred
Copy link
Contributor Author

mildred commented Jan 8, 2016

Also, is there any objection to having two different path syntaxes? One specific for filesystems and one for accessing properties.

An _IPLD merkle-path_ is an extension of a _filesystem merkle-path_ which uses a special syntax to access link properties. **[In case we use escaping in protobuf IPLD format** Except that key escaping is not performed when looking up items in the IPLD objects. This allow accessing reserved keys using _IPLD merkle-paths_ that are not accessible in filesystems.**]**


Path elements are suffixed by either `.link` to access the link properties or by `.object` to dereference the _merkle-link_. if no suffix is present, the _merkle-link_ is dereferenced (to be compatible with _filesystem merkle-paths_ in most cases)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update: Ignore me, it's too early, this is pretty much the second solution I just realized


Disclaimer: I'm late to this party so if I missed something please excuse me

I do very much like the notion of this solution, but .link and .object seem quite verbose to me, why not use two separators like # and . instead, so the example paths would be transformed to the ones you can see below

  • the path /ipfs/QmCCC...000/stuff/foo#mode yields 0755
  • the path /ipfs/QmCCC...000/stuff/foo/other/cat#mode does not exists because other does not have a cat object, only a cat.link
  • the path /ipfs/QmCCC...000/stuff/foo/other/cat.link.mode yields 0644
  • the path /ipfs/QmCCC...000/stuff/foo.other/cat# yields object QmCCC...222

This would make the paths a little less verbose for me. I do see that conflicts with things like cat.link in the second object are still an issue, so it might make sense to use different separators, though . feels quite natural for the object selection to me.

@dignifiedquire
Copy link
Member

I prefer the second solution (/ipfs/QmCCC...000/stuff.foo/other.cat\.jpg) because it is

  • less verbose than the first one, and
  • feels more "natural" as I'm used to / being a separator in e.g. http links and . being a separator for nested object notation.

So something like /ipfs/Qm...000/my.object/other.cat is very easy to parse and write mentally for me.

@mildred
Copy link
Contributor Author

mildred commented Jan 8, 2016

I have no strong opinions on the matter, but I just feel that . would be confusing with extensions (as jbenet also notices). Also the \ escape character is difficult to use in some circumstances (when there are multiple layers of those).

@jbenet, do you have a preference for one syntax or the other ?

@jbenet
Copy link
Member

jbenet commented Jan 8, 2016

@mildred still mulling over. I'm re-confronting this once more.

quick note:

Also, is there any objection to having two different path syntaxes? One specific for filesystems and one for accessing properties.

We could do this. my intuition is that it will be quite confusing for people (when to use each), so if we do go down this path we need to explain the clear need for them, have them be distinct enough. looking at the simplicity of JSON is a good reminder of how simple formats win.

It is also possible to allocate a prefix, like with /ipns -- we could have /unixfs/... paths be proper files and /ipfs/... be raw IPLD objects. or /ipfs/... and /ipld/... or whatever. there is already a dichotomy/tension between "unixfs" + "ipfs objects".

@mildred
Copy link
Contributor Author

mildred commented Jan 8, 2016

It is also possible to allocate a prefix, like with /ipns -- we could have /unixfs/... paths be proper files and /ipfs/... be raw IPLD objects. or /ipfs/... and /ipld/... or whatever. there is already a dichotomy/tension between "unixfs" + "ipfs objects".

You're right, we don't need to have different kind of merkle-paths if we do not intend to use them for filesystems for HTTP gateways. If as you say (and I have forgotten) the unixfs paths is already different from merkle-paths, we can decide for just one kind of merkle-paths: what I called IPLD merkle-paths.

@mildred
Copy link
Contributor Author

mildred commented Jan 10, 2016

Probably obsolete in favor of #62

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants