SPARQL Service Description: add "features/dialects" and a client protocol

[VladimirAlexiev]

Why?

@rubensworks (w3c/EasierRDF#88 (comment)) and @HolgerKnublauch (w3c/EasierRDF#88 (comment)) motivate the need to express SPARQL "features" and "dialects" to enable piece-wise extension of SPARQL.

Previous work

• SPARQL Service Description (SD) describes some details on what a SPARQL endpoint provides
• https://developer.mozilla.org/en-US/docs/Learn/Tools_and_testing/Cross_browser_testing/Feature_detection for HTML, CSS, JS features
• HTTP headers Accept, Accept-Language, Accept-Profile (DX-PROF, I think this is upcoming, not part of HTTP yet)
• The recent discussion on RDF-star MIME types (can anyone point to it?) showed various difficulties when you start combining aspects of data formats

Proposed solution

Not a solution, but some considerations:

• An extended SD is the obvious choice to describe what a SPARQL endpoint provides in a lot more detail
• But we also need to devise a client protocol (what the client wants). Should it be based on simple HTTP headers (easier) or some RDF payload (can express more complex features)?
• A catalog of features could be developed from this issue list, but we need more work to turn it into a coordinated set.
  • IMHO one of the best contributions to the development of RDF Shape languages was the catalog of Use Cases that was compiled, I have something like that in mind

Considerations for backward compatibility

This is a strong enabler for backward compatibility, so that new SPARQL features can be introduced and implemented in a piece-wise way, rather than all together in globally coordinated versions (1.2, 1.3, etc)

[rubensworks]

I agree that the SD is a great place to describe these features, so that client can detect these features, and possibly modify their query according to the supported features. (this is in essence how the Comunica query engine handles heterogeneous sources)

If this were to be added, I think it would be important that the SD would be made mandatory: #153.
Otherwise it will become very hard for clients to figure out with certainty what the server supports exactly.

---

Related work:

The TPF interface makes use of a feature-based hypermedia description, which builds upon the Hydra ontology.

[kasei]

sd:feature, sd:languageExtension, and other related terms were designed to support this sort of development. Can you describe an example of wanting to "express SPARQL 'features' and 'dialects'" that wouldn't be supported by the current Service Description terms?

But we also need to devise a client protocol

I don't understand this. What would this client protocol do? Would this be completely orthogonal to the service description (which is accessed through a simple HTTP request)?

[afs]

I would have thought the number one concern in looking at an endpoint is what data it has. Then, if there are choices of endpoints does the question of how to get the data become significant.

There are now related approaches like content-negotiation by profile that play into the mix.

[VladimirAlexiev]

@kasei If we can just instantiate these mechanisms (catalog of features could be developed from this issue list) that would be perfect. But the depth and complexity of ideas discussed here may in some cases require:

• custom sub-properties
• hierarchical structures (think a feature-subfeature tree)
• dependencies (feature X only makes sense over language Y)

Examples:

• your Support window functions #47: is it a sd:Aggregate or not quite?
• just what is our JOIN LATERAL or Correlated Subquery #100? Do we just make it a sd:Feature without further characterization?
• will sparql-star be mapped to sd:Language sd:SPARQL12-star? But how about its variants, are they yet more sd:Languages or are they sd:Features of the same Language?
• sd:BasicFederatedQuery says a repo supports SPARQL Federate.
  • But how to express Wikidata's whitelist https://www.mediawiki.org/wiki/Special:MyLanguage/Wikidata_Query_Service/User_Manual/SPARQL_Federation_endpoints (also https://www.wikidata.org/wiki/Wikidata:SPARQL_federation_input)?
  • And if it has one of the access control flavors add Authentication to Federation #117, do we express this as an independent sd:Feature individual, or somehow subjugate it to sd:BasicFederatedQuery?

BTW, https://gist.github.com/VladimirAlexiev/8c861db09c08ff202311bae2ee0d6b30 gets the extra 3 flavors of individuals for SD (noticed some problems and emailed Ivan).

What would this client protocol do? Would this be completely orthogonal to the service description (which is accessed through a simple HTTP request)?

Ideally, they should be the same for consistency.
But client "accept" instructions have a tradition:

• they are Accept-* headers with custom and relatively simple structure
• they express preference order (the q parameter), which a SD doesn't have (it's a "set" of things that the server offers)

I don't think a client will necessarily be happy to send a bunch of RDF just to state what he prefers.
In other words:

• SD servers want to speak comprehensively and understandably, i.e. with RDF
• But clients may prefer to speak in shorthand, using a simple DSL

@afs Agree, DX-PROF is very relevant.

[VladimirAlexiev]

• describe optimizer hints in SPARQL Service Description #181
• Extend the SPARQL Service Description with prefix declarations #134
• Easier addition of support for custom datatypes to SPARQL endpoints #130 asks for easier ways to extend SPARQL servers with datatypes.
  • Describing datatypes in SD is very relevant eg for GeoSPARQL that captures geometries in special datatyped literals: geo:wktLiteral, geo:gmlLiteral etc.
  • SD should describe sd:extensionFunction, extensionAggregate, propertyFeature opengeospatial/ogc-geosparql#481 discusses how to elaborate the SD of GeoSPARQL