DSP + dcat:Distribution + dataAddress

[galanisd]

Now in edc 0.2.1, 0.3.1 and 0.5.1 (checking that now) an asset I think can be linked only with one actual dataset; i.e., via an DataAddress.

According to DSP each dataset may contain more than one distribution. See below

https://docs.internationaldataspaces.org/ids-knowledgebase/v/dataspace-protocol/catalog/catalog.protocol#id-1.1.2-distributions
==>

A Dataset may contain 0..N DCAT Distributions. Each distribution must have at least one DataService which specifies where the distribution is obtained. Specifically, a DataService specifies the endpoint for initiating a Contract Negotiation and Transfer Process.

Also according to DSP each distribution must have at least one DataService which specifies where the distribution is obtained (=downloaded?).

Does that mean that multiple DataAddress(es) per asset will be supported in the future or in general that it will possible to link
somehow (via DataAddress or dcat:distribution in metadata or both) an asset with multiple actual data; e.g. dataInXML.zip and dataInText.zip ?

[jimmarino]

Where a distribution is obtained is not the same as downloaded:

Each distribution must have at least one DataService which specifies where the distribution is obtained. Specifically, a DataService specifies the endpoint for initiating a Contract Negotiation and Transfer Process.

An asset's DataAddress is not the same as its distribution. The asset's DataAddress is a reference to where the asset contents are that is scoped to the connector and not dereferenceable externally. A single DataAddress may be used to dereference asset contents and make them available via different distributions, for example, over an HTTP protocol or S3.

[pennyl67]

So, if I understand well, "Distribution" is used for different access protocols (e.g. HTTP protocol or S3)?
According to DCAT, a Distribution is "A specific representation of a dataset. A dataset might be available in multiple serializations, [...] including [...] format". In this case, how can we refer to the same dataset available in CSV and plain text format? Both available with the same access protocol as two different zip files and accessible at different URLs?

[jimmarino]

A distribution can be used to model both the available protocol and a particular format type.

[galanisd]

We have upgraded to edc 0.6.2 (currently the latest tag) and tested a few scenarios.
We noticed the following:
(As far as I remember earlier versions of edc were doing the same things).

Scenario 1: We created an asset without any dcat:distribution in the properties. We created a contract for it by associating with a policy (access + contract). We called get catalog. EDC creates automatically a "dcat:distribution with 3 dcat:Distribution.

			"dcat:distribution": [
				{
					"@type": "dcat:Distribution",
					"dct:format": {
						"@id": "HttpProxy-PUSH"
					},
					"dcat:accessService": {
						"@id": "2a28ddcb-339e-4f05-85fb-e6f557465092",
						"@type": "dcat:DataService",
						"dct:terms": "connector",
						"dct:endpointUrl": "http://localhost:8282/api/v1/ids"
					}
				},
				{
					"@type": "dcat:Distribution",
					"dct:format": {
						"@id": "HttpData-PUSH"
					},
					"dcat:accessService": {
						"@id": "2a28ddcb-339e-4f05-85fb-e6f557465092",
						"@type": "dcat:DataService",
						"dct:terms": "connector",
						"dct:endpointUrl": "http://localhost:8282/api/v1/ids"
					}
				},
				{
					"@type": "dcat:Distribution",
					"dct:format": {
						"@id": "AmazonS3-PUSH"
					},
					"dcat:accessService": {
						"@id": "2a28ddcb-339e-4f05-85fb-e6f557465092",
						"@type": "dcat:DataService",
						"dct:terms": "connector",
						"dct:endpointUrl": "http://localhost:8282/api/v1/ids"
					}
				}
			],

I had a look into the code. It creates one dcat:Distribution per allowed destination type of the dataplane based on our config.
(edc.dataplane.selector.defaultplane.destinationtypes=HttpData,HttpProxy,AmazonS3).
Question 1: Currently I do not think that these distributions are used somehow from the consumer EDC. Or not?
Are there any plans for using them in the future?

Scenario 2: We created an asset with a dcat:distributionin the properties

  "dcat:distribution": [
     {
       "@type": "dcat:Distribution",
       "ms:size": [
         {
           "ms:amount": 100,
           "ms:sizeUnit": {
             "@id": "ms:sentence1"
           }
         },
         {
           "ms:amount": 200,
           "ms:sizeUnit": {
             "@id": "ms:translationUnit"
           }
         }
       ],
       "dcat:packageFormat": {
         "@id": "szip",
         "@type": "dct:MediaType"
       },
       "dcat:byteSize": 300254687,
       "dcat:downloadURL": {
         "@id": "http//www.example.com"
       },
       "dcat:accessURL": {
         "@id": "http//www.example.com"
       },
       "dcat:mediaType": {
         "@id": "http://www.iana.org/assignments/media-types/application/zip"
       }
     }
   ]

We created a contract for the asset by associating with a policy (access + contract). We called get catalog.
the respective dcat:Dataset contains our dcat:distribution not the one that is generated by EDC (somehow ours replaces the EDC generated).

Question 2: Since ours is kept this is not an issue for us. It is a bug that EDC-generated is not also kept?

Attaching the catalog response for both assets (Scenario 1 & 2)
catalogrequestresult.json