Add Appendix A on sato examples #253

y-guyon · 2024-09-26T13:49:08Z

Fixes #252

y-guyon · 2024-09-26T13:51:16Z

index.bs

@@ -406,7 +407,7 @@ No color space conversion, matrix coefficients, or transfer characteristics func

 The output reconstructed image is made up of the output samples, whose values shall be each clamped to fit in the number of bits per sample as defined by the <code>'[=pixi=]'</code> property of the reconstructed image item. The <code>[=full_range_flag=]</code> field of the <code>'[=colr=]'</code> property of <code>[=colour_type=]</code> <code>'[=nclx=]'</code> also defines a range of values to clamp to, as defined in [[!CICP]].

-<div class="example">An 8-bit primary [=MIAF image item=] can be combined with another 8-bit hidden [=MIAF image item=], both used as input image items to a [=Sample Transform Derived Image Item=], with an expression corresponding to ReconstructedSample = 256 × PrimarySample + HiddenSample. The primary [=MIAF image item=] and the [=Sample Transform Derived Image Item=] are both part of the same <code>'[=AVIF/altr=]'</code> group. This will be perceived as a backward-compatible regular 8-bit image to readers not supporting [=Sample Transform Derived Image Items=], and can be decoded as a 16-bit image otherwise, making that pattern a bit-depth extension mechanism.</div>
+NOTE: <a href="#sato-examples">Appendix A</a> contains examples of Sample Transform Derived Image Item usage.


I was not able to make [Appendix A](#sato-examples) work despite being used in bikeshed doc:

https://github.com/speced/bikeshed/blob/828e63d35520ceba577f75b2d88ee92560d4bd58/docs/index.bs#L2631

<a> seems to give the same result anyway.

y-guyon · 2024-09-26T13:51:47Z

index.bs

@@ -659,7 +660,7 @@ The result of any computation underflowing or overflowing the intermediate bit d

 [=Sample Transform Derived Image Items=] use the postfix notation to evaluate the result of the whole expression for each reconstructed image item sample.

-  - <assert>The [=sato/tokens=] shall be evaluated in the order they are defined in the metadata (the <code>SampleTransform</code> structure) of the [=Sample Transform Derived Image Item=].</assert>
+  - <assert>The [=sato/tokens=] shall be evaluated in the order they are defined in the metadata (the <code><dfn export>SampleTransform</dfn></code> structure) of the [=Sample Transform Derived Image Item=].</assert>


Not sure about exporting this struct name.

Yeah, I was debating whether to add it or not.

y-guyon · 2024-09-26T13:55:52Z

index.bs

+
+NOTE: The first input image item supports reasonable loss during encoding because the second input image item "overlaps" by 4 bits to correct it. The second input image item supports reasonable loss during encoding.
+
+NOTE: This pattern can be used for reconstructed bit depths beyond 16 by combining more than two input image items or with various input bit depth configurations and operations.


For the title tree to make sense, there should be a section on floating point recipes here, but this is likely too involved.

Yes, the floating point recipes are complicated. I haven't reviewed them.

I think it's fine to leave them out for now. We may have other examples in the future that are not for bit depth extension. Having the title tree anticipate that makes it easier to add.

y-guyon · 2024-09-26T13:56:40Z

index.bs

+
+<h4 id="sato-example-suffix-bit-depth-extension">Suffix bit depth extension</h4>
+
+The following example describes how to leverage a [=Sample Transform Derived Image Item=] on top of a regular 8-bit [=MIAF image item=] to extend the decoded bit depth to 16 bits.


It would be more accurate to put this whole section in a <div class="example"> but I doubt it will work.

leo-barnes · 2024-09-27T08:13:54Z

I don't love how appendices are rendered in Bikeshed, but there doesn't really seem to be any alternative. I've filed this issue to see if it's possible to get support for lettered annexes: speced/bikeshed#2929

index.bs

leo-barnes

I played around with various ways of trying to get a Letter.Number Annex tree, but it does not seem to be supported. I think this is the best we can do for now until Bikeshed implements something like speced/bikeshed#2929

y-guyon · 2024-09-27T12:35:19Z

Rebased.

wantehchang

Yannis: Thank you for adding this appendix. I suggest some changes.

wantehchang · 2024-09-27T17:20:07Z

index.bs

@@ -697,12 +698,12 @@ The '<dfn for="AVIF">ster</dfn>' entity group as defined in [[!HEIF]] may be use
 The brand to identify [=AV1 image items=] is <dfn export for="AVIF Image brand">avif</dfn>.

 Files that indicate this brand in the <code>[=compatible_brands=]</code> field of the <code>[=FileTypeBox=]</code> shall comply with the following:
-    - <assert>The primary item shall be an [=AV1 Image Item=] or be a derived image that references directly or indirectly one or more items that all are [=AV1 Image Items=].</assert>
+    - <assert>The [=primary image item=] shall be an [=AV1 Image Item=] or be a derived image that references directly or indirectly one or more items that all are [=AV1 Image Items=].</assert>


The term "primary item" is correct. It comes from the PrimaryItemBox ('pitm'). It is used in ISOBMFF, HEIF, and MIAF.

The term "primary image item" is used in HEIF and MIAF and occurs only four and five times respectively (much less frequent than "primary item").

It seems that these standards use these two terms interchangeably.

wantehchang · 2024-09-27T17:34:22Z

index.bs

+
+<h2 id="sato-examples">Appendix A: Sample Transform Derived Image Item Examples</h2>
+
+This informative appendix contains demonstration recipes for extending base AVIF features with [=Sample Transform Derived Image Items=].


Nit/optional: Change "demonstration" to "example"?

wantehchang · 2024-09-27T17:36:50Z

index.bs

+
+NOTE: If the first input image item is the [=primary image item=] and is enclosed in an <code>'[=AVIF/altr=]'</code> group with the derived image item, the first input image item is also a backward-compatible 12-bit regular coded image item that can be used by decoding contexts that do not support [=Sample Transform Derived Image Items=] or do not need extra precision.
+
+NOTE: The first input image item supports reasonable loss during encoding because the second input image item "overlaps" by 4 bits to correct it. The second input image item supports reasonable loss during encoding.


Nit/optional: change "correct it" to "correct the loss"

wantehchang · 2024-09-27T17:45:59Z

index.bs

+<h4 id="sato-example-residual-bit-depth-extension">Residual bit depth extension</h4>
+
+The following example describes how to leverage a [=Sample Transform Derived Image Item=] on top of a regular 12-bit [=MIAF image item=] to extend the decoded bit depth to 16 bits.<br>
+It differs from the <a href="sato-example-suffix-bit-depth-extension">Suffix bit depth extension example</a> by its slightly longer series of operations allowing its first input image item to be lossily encoded.


Nit: don't capitalize "suffix".

wantehchang · 2024-09-27T17:52:29Z

index.bs

+
+Consider the following:
+    - A [=MIAF image item=] being a lossily coded image item,<br>and its <code>'[=pixi=]'</code> property with <code>[=bits_per_channel=]</code>=12,
+    - Another [=MIAF image item=] being a lossily or losslessly coded image item with the same dimensions and number of samples as the first input image item,<br>and its <code>'[=pixi=]'</code> property with <code>[=bits_per_channel=]</code>=8,<br>with the following contraints:


contraints => constraints

wantehchang · 2024-09-27T17:53:11Z

index.bs

+Consider the following:
+    - A [=MIAF image item=] being a lossily coded image item,<br>and its <code>'[=pixi=]'</code> property with <code>[=bits_per_channel=]</code>=12,
+    - Another [=MIAF image item=] being a lossily or losslessly coded image item with the same dimensions and number of samples as the first input image item,<br>and its <code>'[=pixi=]'</code> property with <code>[=bits_per_channel=]</code>=8,<br>with the following contraints:
+        - For each sample position in each plane,<br><math><msub><mi>sample</mi><mi>original</mi></msub></math> being the value of the 16-bit original sample at that position in that plane,<br><math><msub><mi>sample</mi><mi>1</mi></msub></math> being the value of the 12-bit sample of the first input image at that position in that plane,<br><math><msub><mi>sample</mi><mi>2</mi></msub></math> being the value of the sample of the second input image at that position in that plane,<br><math><mo>≈</mo></math> representing similarity within compression loss range,


Nit: In this sentence, I think "being" can all be changed to "is", and "representing" can be changed to "represents".

wantehchang · 2024-09-27T18:01:15Z

index.bs

+            - <code>[=sato/token=]</code>=2 (sample from 2<sup>nd</sup> input image item)
+            - <code>[=sato/token=]</code>=128 (sum)
+            - <code>[=sato/token=]</code>=0, <code>[=sato/constant=]</code>=-127
+            - <code>[=sato/token=]</code>=128 (sum)


IMPORTANT: These two lines (1268-1269) mean adding the constant -127. It does not match the postfix and infix notations below, which subtract the constant 128.

wantehchang · 2024-09-27T18:05:30Z

index.bs

+            - <math><msub><mi>sample</mi><mi>2</mi></msub><mo>≈</mo><msub><mi>sample</mi><mi>original</mi></msub><mo>-</mo><msup><mn>2</mn><mn>4</mn></msup><mo>×</mo><msub><mi>sample</mi><mi>1</mi></msub><mo>+</mo><msup><mn>2</mn><mn>7</mn></msup></math>
+            - <math><mn>0</mn><mo>≤</mo><msub><mi>sample</mi><mi>1</mi></msub><mo>&lt;</mo><msup><mn>2</mn><mn>12</mn></msup></math>
+            - <math><mn>0</mn><mo>≤</mo><msub><mi>sample</mi><mi>2</mi></msub><mo>&lt;</mo><msup><mn>2</mn><mn>8</mn></msup></math>
+            - <math><mn>0</mn><mo>≤</mo><msup><mn>2</mn><mn>4</mn></msup><mo>×</mo><msub><mi>sample</mi><mi>1</mi></msub><mo>+</mo><msub><mi>sample</mi><mi>2</mi></msub><mo>-</mo><msup><mn>2</mn><mn>7</mn></msup><mo>&lt;</mo><msup><mn>2</mn><mn>16</mn></msup></math>


I remember I asked about this when I reviewed an earlier version of the Sample Transform proposal. If sample_1 and sample_2 are both equal to 0, they satisfy the first two inequalities (lines 1256 and 1257) but do not satisfy this inequality. I wonder if we need to clamp the expression in the middle of this inequality to the range [0, 2^16).

I wonder if we need to clamp the expression in the middle of this inequality to the range [0, 2^16).

Good point. This will happen automatically though since the sato language says that the final result is clamped to the output bit-depth. But it might be good to call it out explicitly.

wantehchang · 2024-09-27T18:12:30Z

index.bs

+
+NOTE: The first input image item supports reasonable loss during encoding because the second input image item "overlaps" by 4 bits to correct it. The second input image item supports reasonable loss during encoding.
+
+NOTE: This pattern can be used for reconstructed bit depths beyond 16 by combining more than two input image items or with various input bit depth configurations and operations.


Yes, the floating point recipes are complicated. I haven't reviewed them.

y-guyon commented Sep 26, 2024

View reviewed changes

leo-barnes reviewed Sep 27, 2024

View reviewed changes

index.bs Outdated Show resolved Hide resolved

leo-barnes approved these changes Sep 27, 2024

View reviewed changes

y-guyon added 2 commits September 27, 2024 14:34

Add Appendix A on sato examples

bbb7d70

Qualify appendix as informative

a97eb9f

y-guyon force-pushed the sato_examples branch from ea98221 to a97eb9f Compare September 27, 2024 12:35

y-guyon requested a review from wantehchang September 27, 2024 12:35

wantehchang reviewed Sep 27, 2024

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Appendix A on sato examples #253

Add Appendix A on sato examples #253

y-guyon commented Sep 26, 2024 •

edited by pr-preview bot

Loading

y-guyon Sep 26, 2024

y-guyon Sep 26, 2024

leo-barnes Sep 27, 2024

y-guyon Sep 26, 2024

wantehchang Sep 27, 2024

leo-barnes Sep 27, 2024

y-guyon Sep 26, 2024

leo-barnes commented Sep 27, 2024

leo-barnes left a comment

y-guyon commented Sep 27, 2024

wantehchang left a comment

wantehchang Sep 27, 2024

wantehchang Sep 27, 2024

wantehchang Sep 27, 2024

wantehchang Sep 27, 2024

wantehchang Sep 27, 2024

wantehchang Sep 27, 2024

wantehchang Sep 27, 2024

wantehchang Sep 27, 2024

leo-barnes Sep 27, 2024

wantehchang Sep 27, 2024


		NOTE: The first input image item supports reasonable loss during encoding because the second input image item "overlaps" by 4 bits to correct it. The second input image item supports reasonable loss during encoding.

		NOTE: This pattern can be used for reconstructed bit depths beyond 16 by combining more than two input image items or with various input bit depth configurations and operations.


		<h4 id="sato-example-suffix-bit-depth-extension">Suffix bit depth extension</h4>

		The following example describes how to leverage a [=Sample Transform Derived Image Item=] on top of a regular 8-bit [=MIAF image item=] to extend the decoded bit depth to 16 bits.


		<h2 id="sato-examples">Appendix A: Sample Transform Derived Image Item Examples</h2>

		This informative appendix contains demonstration recipes for extending base AVIF features with [=Sample Transform Derived Image Items=].


		NOTE: If the first input image item is the [=primary image item=] and is enclosed in an <code>'[=AVIF/altr=]'</code> group with the derived image item, the first input image item is also a backward-compatible 12-bit regular coded image item that can be used by decoding contexts that do not support [=Sample Transform Derived Image Items=] or do not need extra precision.

		NOTE: The first input image item supports reasonable loss during encoding because the second input image item "overlaps" by 4 bits to correct it. The second input image item supports reasonable loss during encoding.

Add Appendix A on sato examples #253

Are you sure you want to change the base?

Add Appendix A on sato examples #253

Conversation

y-guyon commented Sep 26, 2024 • edited by pr-preview bot Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

leo-barnes commented Sep 27, 2024

leo-barnes left a comment

Choose a reason for hiding this comment

y-guyon commented Sep 27, 2024

wantehchang left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

y-guyon commented Sep 26, 2024 •

edited by pr-preview bot

Loading