add BlockAdditionMapping to describe each BlockAdditional data #287

robUx4 · 2018-11-18T13:59:51Z

This is similar to a TrackID + CodecID + CodecPrivate + TrackName but for extra frame data.

The Timecode and RAWCooked values may be removed and done in a separate spec. They need to define properly how to interpret the BlockAddIDExtraData binary data.

JeromeMartinez · 2018-11-18T14:04:27Z

IMO it should be in https://github.com/Matroska-Org/matroska-specification/pull/276/files directly.

enum restriction: I am currently not in favor to restrict the list, as we don't limits e.g. tags.

robUx4 · 2018-11-18T14:10:25Z

IMO it should be in https://github.com/Matroska-Org/matroska-specification/pull/276/files directly.

What do you mean by "it" ? As discussed on CELLAR putting a name and differentiate string/binary for something that can already be done with BlockAdditional doesn't seem like the right way to go. The lace number may be an issue, but it can certainly be added to BlockMore.

enum restriction: I am currently not in favor to restrict the list, as we don't limits e.g. tags.

The enum is open-ended, there can be other values. But if I'm going to mention that, then I'll probably remove Timecode and RAWCooked to mention that currently only value 0 is defined.

JeromeMartinez · 2018-11-18T14:27:32Z

What do you mean by "it" ?

OK, I am lost now.
I understand that both PRs resolves half of the request, so IMO both PRs should be merged in one and discussed in a single place.
This PR alone is not useful, if I understand it well, because there isn't the associated Block part.

robUx4 · 2018-11-18T14:42:31Z

This PR solves the current situation where there are BlockAddID and BlockAdditional but it doesn't say what to do with the data. Now we say the current use is a complement of data to give to the codec. But there can be other cases of metadata attached to each frame that don't fall in this category.

The PRs are separated. #276 should be updated to use this new system (thus it comes after).

t-rapp · 2018-11-19T15:56:00Z

The enum is open-ended, there can be other values. But if I'm going to mention that, then I'll probably remove Timecode and RAWCooked to mention that currently only value 0 is defined.

I would appreciate it if Timecode would not be mentioned here. It looks like a premature standardization of timecode storage format to me. Rather allow users to make their own BlockAddIDType definition (if really necessary) possibly with some reserved range for later standardization.

mjbshaw · 2018-11-20T19:08:47Z

It would be nice to clarify how new BlockAddIDTypes can be registered. We at Google are considering adding some new side metadata for frames within WebM (using BlockAdditions). I don't have any concrete details I can share about this metadata right now, but it would be nice to know what the procedure is for standardizing it so we (and others) can utilize that procedure.

dericed · 2018-11-25T21:09:45Z

enum restriction: I am currently not in favor to restrict the list, as we don't limits e.g. tags.

The enum is open-ended, there can be other values. But if I'm going to mention that, then I'll probably remove Timecode and RAWCooked to mention that currently only value 0 is defined.

The EBML specification implies that enum lists are restrictive, not open. https://github.com/Matroska-Org/ebml-specification/blob/master/specification.markdown#restriction-element

ebml_matroska.xml

notes.md

dericed · 2018-11-25T22:23:08Z

Hope you don't mind me working in the same branch @robUx4. I added a few commits here to start to define how to define BlockAdditions for Codec Mappings. So far, I think they are used for A_WAVPACK4 (for the lossless part) and VP8/VP9 (for alpha encodings). I haven't found others, but do they exist? Then IIUC correctly, the values of 2 and greater are arbitrarily used to relate BlockAdditional to the new BlockAddition mappings.

robUx4

That's what collaboration is for :)
I don't think there are other uses of BlockAdditions but then I forgot about VPx alpha...

codec_specs.md

ebml_matroska.xml

dericed · 2019-07-30T02:58:47Z

@robUx4 can you review the approach I'm testing in 3d5e52d. This moves the Block Additional Mappings to be handled similarly to Codec Mappings, rather than enumerating them in the schema.

This is similar to a TrackID + CodecID + CodecPrivate + TrackName but for extra frame data.

…ificied value

dericed · 2019-09-14T21:38:50Z

@robUx4 I updated this PR to start a block_additional_mapping (timecode as a string in this example) as a standalone document. I think the timecode document could be expanded to allow BlockAddIDExtraData to support different types of timecode such as expressions of SMPTE 12M or QuickTime's timecode descriptor of QuickTime Timecode Sample Data via https://github.com/cellar-wg/matroska-specification/pull/112/files.

However in this PR, timecode is mostly here as a first example while the PR more so focuses on the Block Additional Mapping section and how to define the mappings.

notes.md

mjbshaw · 2019-09-24T19:57:30Z

I can finally talk about what I alluded to earlier. YouTube now ingests and serves HDR10+ dynamic metadata. BlockAddID == 4 is used for VP9 to signal that the BlockAdditional payload is ITU T.35 metadata as defined by ITU-T T.35 terminal codes. I think it might make sense to have some kind of "registry" or something for BlockAddIDType. Without that I think various muxer implementations will diverge in the IDs they use for various types of metadata.

dericed · 2019-10-12T18:33:36Z

Hi @robUx4, I did a little redesign of this PR so now there's a Block Additional Mapping introduction in its own document with an example in XML. In the Makefile this is add along with all markdown within the block_additional_mapping directory into the end of the Codec document. Please re-review.

dericed · 2019-10-12T18:34:59Z

My recent updates don't accommodate @mjbshaw's comment at #287 (comment) as I think that would be better handled in a later PR as we'll have to discuss reserving BlockAddID=4.

ebml_matroska.xml

block_additional_mappings/timecode.md

robUx4 · 2019-10-13T12:57:54Z

I can finally talk about what I alluded to earlier. YouTube now ingests and serves HDR10+ dynamic metadata. BlockAddID == 4 is used for VP9 to signal that the BlockAdditional payload is ITU T.35 metadata as defined by ITU-T T.35 terminal codes. I think it might make sense to have some kind of "registry" or something for BlockAddIDType. Without that I think various muxer implementations will diverge in the IDs they use for various types of metadata.

Thanks for the tip @mjbshaw !
I think we should handle independant BlockAdditional types (ie not related as separate specs), just as it should be done for codecs (see the AV1 specs). At first we'll probably include a lot of common ones in the main codec specs.

BlockAddID of 1 is already covered by these changes. For the value of 4 it should theoretically be combined with a BlockAddIDValue in the Track header. It seems unlikely YouTube will support that, since the spec is not even finalized yet. So if we want to be coherent we should treat value 4 as value 1, as a reserved/preexisting value. 1 is codec specific (ie data passed to the decoder with each frame), while 4 seems to be more metadata added to the frame.

MaxBlockAdditionID is not supported by WebM either, so we can't really rely on that to know in advance what BlockAddID will be found in the file or not.

On our side we should probably mention that BlockAddID value 4 is also a reserved value and explain what it does. I created #345 so we don't forget to do it once this PR is merged.

dericed · 2019-10-16T13:55:31Z

I reviewed the recent changes and this seems ok to me. There is more work to do on timecode; however that isn't the focus on this PR. I suggest to merge and then we'll follow up with both timecode refinement and HDR.

robUx4

Some adjustments are needed. Or we can split this PR to cover the specific case of timecode BlockAddition separately.

ebml_matroska.xml

block_additional_mappings/timecode.md

since 0 is reserved for main Block and 1 is for codec context

also use new lines (if there's no line in between it produces the same output and is more readable)

and use new lines

…itional Mapping

VP8/VP9 might contain other BlockMore elements in the same BlockAdditions (HDR dynamic metadata)

dericed · 2019-10-22T15:36:28Z

This PR is ready for a review. I removed the timecode block additional mapping (which was here as an example) to separate the discussions, but this focuses on the new elements for block additional mappings and how they are defined.

robUx4 · 2019-10-22T18:47:20Z

LGTM

dericed · 2019-10-22T18:48:28Z

Merging then :)

It was added in ietf-wg-cellar#390 but that value is not allowed since ietf-wg-cellar#287. The Opaque data (used by WavPack) don't make use of that value since it's for data unknown to the codec.

It was added in #390 but that value is not allowed since #287. The Opaque data (used by WavPack) don't make use of that value since it's for data unknown to the codec.

It was added in ietf-wg-cellar#390 but that value is not allowed since ietf-wg-cellar#287. The Opaque data (used by WavPack) don't make use of that value since it's for data unknown to the codec.

robUx4 added the format addition label Nov 18, 2018

robUx4 force-pushed the block_addition_mapping branch from 2c6fd74 to cc2d785 Compare November 18, 2018 14:40

dericed reviewed Nov 25, 2018

View reviewed changes

ebml_matroska.xml Outdated Show resolved Hide resolved

ebml_matroska.xml Outdated Show resolved Hide resolved

notes.md Outdated Show resolved Hide resolved

dericed mentioned this pull request Nov 25, 2018

add block metadata structure #276

Closed

dericed force-pushed the block_addition_mapping branch from cc2d785 to e6d2191 Compare November 25, 2018 22:20

robUx4 commented Dec 2, 2018

View reviewed changes

codec_specs.md Outdated Show resolved Hide resolved

codec_specs.md Outdated Show resolved Hide resolved

codec_specs.md Outdated Show resolved Hide resolved

ebml_matroska.xml Outdated Show resolved Hide resolved

robUx4 added the spec_main Main Matroska spec document target label Dec 18, 2018

mcr assigned dericed Aug 27, 2019

robUx4 and others added 3 commits September 14, 2019 16:33

add BlockAdditionMapping to describe each BlockAdditional data

4ba2d11

This is similar to a TrackID + CodecID + CodecPrivate + TrackName but for extra frame data.

add a specification note about BlockAdditionMapping and the only spec…

68df9c7

…ificied value

define codec-blockadditions for codec mapping and add wavpack + vp89

2c2c348

dericed force-pushed the block_addition_mapping branch 2 times, most recently from e4d7c12 to 74294a2 Compare September 14, 2019 21:35

robUx4 commented Sep 24, 2019

View reviewed changes

notes.md Outdated Show resolved Hide resolved

robUx4 commented Sep 24, 2019

View reviewed changes

notes.md Outdated Show resolved Hide resolved

robUx4 commented Oct 13, 2019

View reviewed changes

ebml_matroska.xml Outdated Show resolved Hide resolved

robUx4 commented Oct 13, 2019

View reviewed changes

block_additional_mappings/timecode.md Outdated Show resolved Hide resolved

robUx4 mentioned this pull request Oct 13, 2019

Support WebM HDR medata support in BlockAdditions #345

Closed

dericed mentioned this pull request Oct 19, 2019

WIP add N_QUICKTIME #102

Closed

robUx4 commented Oct 20, 2019

View reviewed changes

ebml_matroska.xml Outdated Show resolved Hide resolved

block_additional_mappings/timecode.md Outdated Show resolved Hide resolved

block_additional_mappings/timecode.md Outdated Show resolved Hide resolved

dericed and others added 13 commits October 22, 2019 11:34

restrict BlockAddIDValue to 2 or greater

e0455d5

since 0 is reserved for main Block and 1 is for codec context

clarify reserved BlockAdditionID values

3a33717

move/expand note on BlockAdditionalMapping to codec-specs

8e360c0

rewrites to Block Additional Mapping

006fab1

add Block Additional Mappings to Codec document

84b03dd

fix incorrect block-additional-mapping link

fc0607e

document the value 1 of BlockAddID in the BlockAddID description

61d8fb6

backtick A_WAVPACK4 in the Codec BlockAdditions description

d9efc52

use similar phrasing for BlockAddID=1 and BlockAddID>1

d160aef

also use new lines (if there's no line in between it produces the same output and is more readable)

simplify the text to tell BlockAddIDName is optional

8725a30

and use new lines

rephrase the "additional contextual information" use in the Block Add…

e5663de

…itional Mapping

mention the BlockAddId is 1 for each Codec BlockAdditions

aedce3d

VP8/VP9 might contain other BlockMore elements in the same BlockAdditions (HDR dynamic metadata)

fix line lengths in xml comments

dc368ae

dericed force-pushed the block_addition_mapping branch from cfae516 to dc368ae Compare October 22, 2019 15:34

dericed merged commit 87ffd39 into master Oct 22, 2019

dericed deleted the block_addition_mapping branch February 23, 2021 20:53

robUx4 mentioned this pull request Apr 4, 2021

support frame side data #270

Closed

robUx4 mentioned this pull request Dec 1, 2024

remove BlockAddIDValue value for opaque data #922

Merged

robUx4 added a commit that referenced this pull request Dec 8, 2024

remove BlockAddIDValue value for opaque data

860737c

It was added in #390 but that value is not allowed since #287. The Opaque data (used by WavPack) don't make use of that value since it's for data unknown to the codec.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

add BlockAdditionMapping to describe each BlockAdditional data #287

add BlockAdditionMapping to describe each BlockAdditional data #287

robUx4 commented Nov 18, 2018

JeromeMartinez commented Nov 18, 2018

robUx4 commented Nov 18, 2018

JeromeMartinez commented Nov 18, 2018

robUx4 commented Nov 18, 2018

t-rapp commented Nov 19, 2018 •

edited

Loading

mjbshaw commented Nov 20, 2018

dericed commented Nov 25, 2018

dericed commented Nov 25, 2018

robUx4 left a comment

dericed commented Jul 30, 2019

dericed commented Sep 14, 2019

mjbshaw commented Sep 24, 2019

dericed commented Oct 12, 2019

dericed commented Oct 12, 2019

robUx4 commented Oct 13, 2019

dericed commented Oct 16, 2019

robUx4 left a comment

dericed commented Oct 22, 2019

robUx4 commented Oct 22, 2019

dericed commented Oct 22, 2019

add BlockAdditionMapping to describe each BlockAdditional data #287

add BlockAdditionMapping to describe each BlockAdditional data #287

Conversation

robUx4 commented Nov 18, 2018

JeromeMartinez commented Nov 18, 2018

robUx4 commented Nov 18, 2018

JeromeMartinez commented Nov 18, 2018

robUx4 commented Nov 18, 2018

t-rapp commented Nov 19, 2018 • edited Loading

mjbshaw commented Nov 20, 2018

dericed commented Nov 25, 2018

dericed commented Nov 25, 2018

robUx4 left a comment

Choose a reason for hiding this comment

dericed commented Jul 30, 2019

dericed commented Sep 14, 2019

mjbshaw commented Sep 24, 2019

dericed commented Oct 12, 2019

dericed commented Oct 12, 2019

robUx4 commented Oct 13, 2019

dericed commented Oct 16, 2019

robUx4 left a comment

Choose a reason for hiding this comment

dericed commented Oct 22, 2019

robUx4 commented Oct 22, 2019

dericed commented Oct 22, 2019

t-rapp commented Nov 19, 2018 •

edited

Loading