OCM WebDAV service

[ishank011]

What?

The Open Cloud Mesh (OCM) is a protocol that enables sharing of data across various cloud providers, given that they agree on the APIs that they'll be exposing to users from these other providers. For different reva deployments, each of which might connect to different storage backends and authentication providers, this is supposed to work out of the box.

The workflow

Invitation workflow

1. Suppose user Alice at CERN wants to share data with Bob at CESNET. Both the reva deployments at these two organizations will come to know of the other sites by either reading from a json config file or connecting to a central database, we support both the options. Let's say for now that both the sites have the json files with the following content:

[
	{
		"name": "cernbox",
		"services": [
			{
				"endpoint": {
					"type": {
						"name": "OCM"
					},
					"path": "https://cernbox.cern.ch/ocm/"
				},
			},
			{
				"endpoint": {
					"type": {
						"name": "Webdav"
					},
					"path": "https://cernbox.cern.ch/remote.php/webdav/",
				},
			},
		]
	},
	{
		"name": "oc-cesnet",
		"services": [
			{
				"endpoint": {
					"type": {
						"name": "OCM"
					},
					"path": "https://cesnet.cz/ocm/",
				},
			},
			{
				"endpoint": {
					"type": {
						"name": "Webdav"
					},
					"path": "https://cesnet.cz/remote.php/webdav/"
				},
			},
		]
	}
]

2. Now both the sites know of each other, but not about the individual users they have. So Alice@CERN will send a /invites/forward request to Bob@CESNET, asking Bob if he wants to begin sharing of data.
3. Now if Bob accepts the invite, the reva deployment at CESNET will send some metadata about Bob to CERN and then Alice can search for Bob and begin sharing of data with him.

Note that the invitation workflow is already implemented and not under the scope of this project. I've described it only for developers to get a better understanding of how OCM works.

OCM sharing

1. Now the reva deployment at CERN, and hence Alice, knows about Bob@CESNET. So she'll send a POST /shares request to Reva@CERN with the following parameters:

Type: OCM-Share
Path: /users/alice/myFolder
Permissions: Editor
Sharee: Bob@CESNET

2. Reva at CERN will store in its local database the metadata for this share, but this data will also have to be sent to CESNET. Here comes the tricky part. Reva at CERN doesn't know what authentication mechanism CESNET follows; they might require users to sign in using their email and password, or authenticate via an external ID provider like github. So the requirement for any site to participate in OCM sharing is to expose an unprotected endpoint (one which doesn't require the request sender to be authenticated) /ocm/shares. So Reva@CERN will send a POST /ocm/shares request to Reva@CESNET with the following parameters:

Type: OCM-Share
Path: /users/alice/myFolder
Permissions: Editor
Owner: Alice@CERN
Token: some-token-with-access-to-myFolder

3. When Reva@CESNET gets this request, it'll verify that it comes from a trusted source (we can ignore how this verification is done) and store this metadata in its own local database.
4. Now when Bob@CESNET wants to access the folder /users/alice/myFolder at CERN, it'll use the token it received as part of the request to access it and Reva@CERN should have mechanisms to verify that this token has access to that particular path and allow the request to go through. This request will be send to the Webdav endpoint specified in the config file at the top. So this request would look like

GET https://cernbox.cern.ch/remote.php/webdav/users/alice/myFolder
Token: some-token-with-access-to-myFolder

Current implementation

Looking at the above workflow, step 4 seems simple but how we implement right now is a hack. As part of the Token parameter in the POST /ocm/shares request, we send Alice's identifier token which allows access to Alice's complete namespace and is short-lived.

• So the first issue is that Bob can use the token to access /users/alice/secretFolder and our current implementation would allow it.
• The next issue is that we use short-lived tokens (usually 24 hours), so once this expires, Bob won't be able to use it then.

New design

1. We plan to create a new service and expose that to the share receiver. So in the Token parameter, we won't send Alice's token but generate a random ID and send that as part of the POST /ocm/shares call. So this would look like:

POST /ocm/shares
Type: OCM-Share
Path: /users/alice/myFolder
Permissions: Editor
Owner: Alice@CERN
Token: random-share-id

2. We'll update the Webdav endpoint in the config file to point to this new service. Let's say it's exposed at /ocm/webdav. So now Bob@CESNET will send a request like

GET https://cernbox.cern.ch/ocm/webdav/users/alice/myFolder
Token: random-share-id

There's one other alternative. Since reva@CERN will know what path this share ID points to, the request could skip specifying the path and look like

GET https://cernbox.cern.ch/ocm/webdav/random-share-id

We'll consider the pros and cons of both approaches before implementing.

3. Reva at CERN will lookup its database and see if a share with ID random-share-id exists or not, and if it does, see if the request has the appropriate permissions. For example, if Alice creates a share with Reader permissions and Bob tries to write to the file, this should not be allowed.
4. So with this new service, we got rid of both the drawbacks of the current implemetation.

[ishank011]

Looking at the specific reva services and APIs, the workflow would look like:

Alice --> POST /shares --> CreateOCMShare --> POST /ocm/shares --> CreateOCMCoreShare
            HTTP@             GRPC@                HTTP@                 GRPC@
            CERN              CERN                CESNET                 CESNET

Bob --> GET/PUT /ocm/webdav --> GetOCMShare --> InitiateFileUpload/Download
              HTTP@               GRPC@                GRPC@
              CERN                CERN                 CERN

Relavant code:
/ocm/shares: https://github.com/cs3org/reva/blob/master/internal/http/services/ocmd/shares.go
CreateOCMShare: https://github.com/cs3org/reva/blob/master/internal/grpc/services/gateway/ocmshareprovider.go
CreateOCMCoreShare: https://github.com/cs3org/reva/blob/master/internal/grpc/services/gateway/ocmcore.go
Database for storing shares: https://github.com/cs3org/reva/blob/master/pkg/ocm/share/manager/json/json.go

[ishank011]

How Bob would be able to access the files received by him:

1. He'd call ListReceivedOCMShares on CESNET's reva.
2. He'd call UpdateReceivedOCMShare to accept the share.
3. /Shares would have a new entry myFolder
4. Bob now wants to list the files in myFolder, reva @ CESNET should make a PROPFIND (listing) call to /ocm/webdav at reva @ CERN