Split the specification into parts

[mattgarrish]

Opening the PR for general review and commenting, as I've gone as far as I'm likely to on my own in terms of splitting the document up.

There are now three parts to the specification:

• Part I defines the manifest in general terms for any "digital publication" (I had to introduce this term to avoid confusion, but maybe there's something better)
• Part II is the more familiar concrete implementation of web publications
• Part III will be where we define any requirements for extending

One place where I really noticed my attempts to generalize the manifest failing was with the table of contents and page lists, which depend entirely on there being a primary entry page. I did my best to allow flexibility for now, but as we've defined relationships for these I'm not sure what their actual property names would be. Will need revision in the future.

---

Preview | Diff

[HadrienGardeur]

One place where I really noticed my attempts to generalize the manifest failing was with the table of contents and page lists, which depend entirely on there being a primary entry page.

We could simply reference the rel values that can identify them in readingOrder and resources, getting rid of the rest of it (this would be moved to a separate "Web Publications" document).

[iherman]

General remark: I think if there is an agreement with the general direction, we should merge this ASAP. There may be technical issues, but it would be way better to handle those via bona fide issues.

In the meantime I will add two issues below which may or may not require further discussion.

---

Minor editorial thing:

• 2.2.: "(i.e., prior to be digital publication being processed by a user agent." Closing parenthesis missing, but I am not even sure the sentence is all right

[iherman]

2.2.: says that each profile can define its own canonical manifest. I am not sure I like that. I would think that there is a standard way to do that (like now) and that profiles have the possibility to extend this when they provide new properties that require additional processing (which must be documented of course). This will require some editing on the canonicalization to describe extension points, but that is for later.

B.t.w., the current setup is also problematic with the WebIDL, because the WebIDL essentially represent the canonical format and not the authored one.

[iherman]

3.1 says

A Web Publication is discoverable in one of two ways: resources either include a link to the manifest (via an HTTP Link header or an HTML link element [html]), or the manifest can be loaded directly by a compatible user agent.

If we are talking about a Web publication, then we did discuss and decide that one can discover one via the HTML entry page only. I think we decided not to go into getting the manifest directly for all kinds of reasons (security, non-WP aware browsers, etc). I think that for Web Publications we must keep to that.

I realize that the package document allows a slightly different approach when things are packaged, but that is probably a separate, and it is not clear to me whether a separate package is a module (I am not sure it is).

[iherman]

Semi editorial issue: while we are making a major editing, it may be worth taking care of Richard's remark in #354 (comment): in 2.6.4.3.1 the description should say

auto: indicates that the textual values are explicitly directionally set to the direction of the first character with a strong directionality, following the rules of the Unicode Bidirectional Algorithm [[!bidi]].

[mattgarrish]

it may be worth taking care of Richard's remark in #354 (comment)

Sure, I didn't want to layer even more into this PR, but if that's all that needs to change I'll add.

[mattgarrish]

(i.e., prior to be digital publication being processed by a user agent.

Yes, this should be:

(i.e., prior to the digital publication being processed by a user agent).

[mattgarrish]

If we are talking about a Web publication, then we did discuss and decide that one can discover one via the HTML entry page only.

Right, I'm guilty of not rereading the intro to that part as closely as I was hoping it was just pre-existing content, but looks like a leftover from when we were still deciding about which options to support. I'll edit that out.

[mattgarrish]

2.2.: says that each profile can define its own canonical manifest. ... I would think that there is a standard way to do that (like now) and that profiles have the possibility to extend this

I don't say that, I don't think. I said that the rules for generating the canonical manifest are defined per format, and I'm not sure how we can avoid that.

How can you define common canonicalization rules without defining the content of a format that doesn't yet exist, since you have to be able to harvest info from it?

What we'd probably be looking at is defining is a generic publication format to go with the manifest.

[iherman]

@mattgarrish, on #407 (comment): I think that even if we say "generating the canonical manifest are defined per format" is a bit loose and can lead to problems with interoperability.

Without having in mind all the details, here is what I thought could be done: what canonicalization does is to define mappings from shortcuts to full values. Thinks like:

• values that are strings and should become internationalized strings
• values that are single values and should become arrays
• values that are single values and should be put into a specific type of objects

etc.

What if we do the following:

• we categorize each relevant term according to the actions that have to be done
• the (generic) canonicalization algorithm, which currently has the term names baked in, would be re-written in terms of steps to be performed on certain categories
• there would be a possible extension point where format specific rules can also be added

With such a generic framework we would categorize the generic terms we have now and the canonicalization would then have the same effects as now; if a module adds new metadata values, they would have to add the relevant category, too.

WDYT?

On the practical side: I think we should flag it in the text and (if otherwise everybody is fine with it) merge this PR before getting into this for the next version.

[mattgarrish]

If you mean generalizing the "generating" step, then we can probably make it work. I was thinking you wanted to move the entire lifecycle section into the manifest, and I'm not sure that's feasible given inevitable differences in how the manifest is obtained (e.g., finding from a web pub versus a package).

[iherman]

@mattgarrish yes! My apologies, I wasn't clear...

[mattgarrish]

Okay, then how about we merge this PR and then I can try some more changes? It would be nice to have a clean reset of the diff/preview before tackling more.

[iherman]

@mattgarrish absolutely. This is what I proposed before: merge it and start from there!

[iherman]

That being said: do we want to get an approval from our bosses? This is a significant change...

Cc: @TzviyaSiegman @wareid @GarthConboy

[mattgarrish]

That being said: do we want to get an approval from our bosses?

Ya, that's what I'm hoping for. I don't want to approve my own changes, so are we good to go?

[wareid]

Looks good to me 👍

[TzviyaSiegman]

Thanks @mattgarrish. This looks good.

[iherman]

@mattgarrish I think we are at the point of: 'ship it'! :-)

We can then look at the canon. section separately.