Should it be possible to create orphan resources?

[kjetilk]

@csarven and I had a video call, as it appears that there are certain issues that mainly interest the two of us :-) There are some points of contention that we need to iron out. One question is this:

Should containers that are not contained within a parent container, (eventually up to /) be allowed to be created/exist in Solid?

LDP appears to be OK with this, any container may be its own island, and containment can be given at will.

[kjetilk]

My view is that this doesn't fit well with the hierarchical folder view of Solid, and so every operation must update containment triples of the parent container to include the resource. This relates to the discussion in #62 , #68 , #39 , etc.

@csarven 's proposition is, based on LDP, that PUT and PATCH should not touch containment. A key piece of contention here is a statement from LDP:

5.2.4.1 LDP servers SHOULD NOT allow HTTP PUT to update a LDPC’s containment triples; if the server receives such a request, it SHOULD respond with a 409 (Conflict) status code.

We interprete this rather differently. I'll let @csarven argue for his position, my position is that it just means that any request that contains containment triples (or other server managed triples) should not be merged into the container. I think it doesn't prohibit the server from containment management. If it did, any PUT would result in an orphan, which is fine from a Linked Data perspective, but possibly not from a Solid perspective.

[TallTed]

The LDP server is absolutely permitted, I daresay required, to manage containment triples, including those that result from PUT and POST.

The LDP constrainment is on client manipulation of containment triples, which are supposed to be managed by the server (albeit they may be changed upon request by the client, as when a resource is moved from one container to another).

I can see that the wording in the LDP spec (in which production both I and @timbl participated) is not as clear as it could be. I wish that were easily and quickly rectifiable.

[csarven]

I find the LDP spec taken in its totality to be fairly clear about the roles of POST and DELETE with interaction models and containment management. Methods like PUT and PATCH are neither prescribed at the same level of specificity as POST nor the possibility even implied. PUT and PATCH are at the very least recommended to not effect containment management.

Regarding 5.2.4.1, the request as a whole should not entail containment updates.

Regarding 5.2.7.1:

5.2.7.1 LDP servers are RECOMMENDED to support HTTP PATCH as the preferred method for updating a LDPC's minimal-container triples.

If PUT and PATCH were at all intended for containment management, there'd be far more and clear signals in the spec. There is not even a mention of certain features prescribed through POST also being possible to achieve with PUT and PATCH.

Moreover, suggesting that PUT and PATCH nearly have equivalent affordances as POST also stretches their purpose with respect to RFC 7231. It also makes the whole point of having different methods literally pointless. Why bother with a massively over-engineered solution where a single overloaded method would suffice?

If we assume that LDP just wasn't clear enough, dare I say that those are not just minor issues here and there to rectify but major potholes in the spec.

However, I'd like to think that LDP did a better job than that, and why I've started a proposal along these lines:

#39 (comment)

As I see it, it doesn't conflict with anything and clarifies with a simple extension while allowing us to have and eat vegan cakes.

I'm applying the Occam's razor because the alternative explanation of LDP either doesn't fit with the spec and/or steps on RFC 7231's toes - one certainly wouldn't expect that from a W3C publication coming from Linked Data folks. But, I suppose it is equally possible that I've misunderstood everything.

---

I contend that LDP permits resources to exist without belonging to any container. This is for example clear in cases where the interaction model is not mentioned in client's request:

This specification does not constrain the server's behavior in other cases.

However, a server can just as well reject a particular interaction model or lack thereof:

If any requested interaction model cannot be honored, the server MUST fail the request.

If we want the Solid spec to not overly depend on LDP's interaction models, this is the chance to make that call. If a resource that's originally created with PUT or PATCH but later needs to be listed in a container, it can be done with a subsequent POST (re: RDF merge/append).

Having said that, if resources are desired to always belong to a container, then they should go through POST with preferred interaction model in Link. While I think always linking containers and resources is a reasonable choice for Solid, I also find it unnecessarily restrictive. The spec shouldn't restrict user's resource management but empower it as much as possible. It leaves room for other ways to manage resources a chance to work in The Solid Ecosystem without having to reformulating the role of LDP's containment requirements in Solid.

[kjetilk]

Moreover, suggesting that PUT and PATCH nearly have equivalent affordances as POST also stretches their purpose with respect to RFC 7231. It also makes the whole point of having different methods literally pointless. Why bother with a massively over-engineered solution where a single overloaded method would suffice?
[...]
#39 (comment)

As I see it, it doesn't conflict with anything and clarifies with a simple extension while allowing us to have and eat vegan cakes.

The way I see it, POST is the method to use to be nice, where the server gets a hint through the Slug, and does its best and we can have vegan cake. PUT is the brutal carnivore, where everything has to be consistent, or otherwise the server rejects the request with a 4xx.

So, PUT is for clients to communicate very specific requirements, and where it would rather have the server reject the request unless all requirements are met than try to apply some intelligence to try to make it right. One of the reasons a Solid server should reject a request is, I think, is if the client tries to create an orphan.

Thus, I think it would make PUT and POST sufficiently different to have them both in, also in the case where we say orphans are not allowed.

I think "no orphans" would be an additional constraint on LDP, and thus something that can be relaxed later.

[csarven]

[I don't think containment is unique to LDPC but any LDPR.. perhaps we reframe the issue/title?]

So, PUT is for clients to communicate very specific requirements, and where it would rather have the server reject the request unless all requirements are met than try to apply some intelligence to try to make it right.

Okay, can you make a straw man list all of those assumptions and requirements?

Can we try to be more clear about the following:

• LDPR can exist without containment applied ?

Straight forward. If you want containment, use POST with the interaction model. Don't write more code. Take a long lunch break. If containing all resources is a must have feature, we could extend by stating that if no interaction model in request, it gets rejected.

• LDPR can be contained by the "root" container on the same system ?

If a resource is not intended to be contained, should the system still enforce it by referring to it from the "root" container?

Axiom: server always knows the URI of the "root" container.

Allowing PUT and PATCH to trigger server's containment management entails that information about the parent container needs to be signaled in the request. As you know, one proposed solution is where server inspects the Request-URI for slashes and determines the closest parent. Perhaps using other ways by expressing "containedIn", "isPartOf", "parent".. in headers or ..

I won't repeat myself.. slashes/PUT etc.. increases the overall complexity. If we stick by POST+Slug driven by the interaction model in Link or lack thereof, it is hardly an extension to LDP - other than to clarify some ambiguities. Did I mention less implementation code and maintenance?

[kjetilk]

Yes, you're right, it is not just containers, so I changed the title.

Actually, I think we are jumping too quickly into technical details. I think there is agreement that LDP allows orphans. We also know that we can constrain it. So, we know that there is a technical solution available whether the answer to the question is yes or no. We can decide upon the use of PUT, PATCH, POST, etc later. We also know that if the answer is no, we can change it to yes later.

The answer to the question is more a design question than a technical question, I think. Do we want a design for now that allows orphans?

[csarven]

I would say that until there is a reason to prohibit a set of use cases - what are they? - keep the system flexible so people can do different things with it.

We can always state these design considerations in Best Practices and Guidelines.

[JordanShurmer]

I'm coming in out of the blue, but will give my 2 cents.. I'm starting a Rust impl of a Solid server, so I've been carefully reading the specs and considering how to implement pieces of if.

I agree with @csarven on his previous comment. I see no reason to prohibit "orphaned" resources.. and I find that label a little arbitrary. Why should every LDP Resource be contained in a container in the first place? That seems like an arbitrary assumption. Containers are available if you want Resources grouped together, but there's no need for a resource to be in one by definition.

[JordanShurmer]

However, in regards to the interpretation of 5.2.4.1 in LDP I agree with @kjetilk's interpretation.

A PUT request from a client to a container url would be updating the container itself, and the spec is saying that it must not update the containment triples. It can be used to update other aspects of the container.

You can still POST Resources outside a container though. And you can also PUT Resources outside containers if you choose. That PUT or POST request would be to the Resource's URL not the container's.

[csarven]

The ability to create orphan resources makes it possible for Solid applications to instantiate/create the root container of a storage.

[JordanShurmer]

As I think more about it, there is one major down side to allowing orphaned resources: Users will not necessarily know and have access to everything that exists in their Pod. (this was probably obvious to others, but I just now realized this implication :D )

If a solid client can create a random resource at some random url, then noone would know about that resource except that app that created it. This would take away the ability of the user to fully manage their own data.

Unless I'm mistaken about this, I would strongly support enforcing containment in solid. This would mean that the Solid spec would have to diverge from LDP's spec to say either

1. servers MUST NOT support POST nor creation via PUT on Resources
2. or servers MUST base containment on url hierarchy (thus resolving Is Solid containment strictly hierarchical?  #98 as well...)

Am I misunderstanding things greatly here?

[dmitrizagidulin]

one major down side to allowing orphaned resources: Users will not necessarily know and have access to everything that exists in their Pod.

Agreed, +1.

[csarven]

one major down side to allowing orphaned resources: Users will not necessarily know and have access to everything that exists in their Pod.

As I see it, given that 1) the resource was initially created with the intention of it not being contained in anything (as per LDPC), 2) Location of the created resource given to the requester 3) not later added to any container 4) assuming it is not linked, discoverable or findable from anywhere else.

..yes, I agree, that's "one major down side to allowing orphaned resources."

The point of raising this question:

LDPR can be contained by the "root" container on the same system ?

was to continue exploring..

If a resource is not intended to be contained, should the system still enforce it by referring to it from the "root" container?
Axiom: server always knows the URI of the "root" container.

Bearing in mind that it is completely legit to create a resource that the user simple doesn't want it (immediately) linked (from a container). Use cases around server-based data portability will have other (internal only?) mechanisms to discover uncontained resources.

[JordanShurmer]

Yeah, that's true.. Solid could specify some other means for clients to find every resource not explicitly included in a container. A new property on the profile, or on the profile's pim:storage or something, indicating all the loose resources. This does help with that concern..

I am still of the mind that containment should be based on hierarchy though. As a Solid user this would map well to a typical file system where you interact with files within folders, as a client application you wouldn't have to worry how a resource was created (only where), and as a server you don't have to manage containment at all - it's just whatever is in the folder.

It would also make for a more clear understanding of what you have in your pod. Consider the following: I grant app X access to create resources under the container /app/x.. I then want to discover everything they've created. The obvious approach of listing everything in that container would not work.

Again, just my thoughts on the issue. Thanks for doing this work Open Source and for the dialog thus far.

[csarven]

Proposal following F2F meeting of 2019-10-29:

Orphan resources makes it challenging for discovery eg. data portability for client applications.

It also adds requirements to WAC system's inheritance algorithm to mitigate loss of control.

Turning orphan resources into non-orphan resources by way of updating containment triples would require additional steps.

It potentially introduces non-uniform URI patterns.

Given that containment based resource management is a central requirement for majority of the use cases, introducing alternatives such as the orphan resources complicates the server implementation.

Hence there are no orphan resources.

[timbl]

It is impossible to have an orphan resource because of #98