How is the format of the decoded frame determined?

[lesomnus]

I'm trying to decode a video encoded with H.264. On the desktop, it is decoded as I444, but on mobile device (Galaxy S10), it is decoded as RGBA. Perhaps the hardware decoder (on mobiel device) only supports RGBA output for efficiency?

1. How is the format of the decoded frame determined?
2. Is there a way to determine whether the output format is I444 or RGBA before decoding?
3. Are there currently any properties in the standard to specify the output format? If not, is there a possibility of adding this in future standards?

[sandersdan]

There is no standardized requirement for the output format from decoding, nor does WebCodecs provide a prediction. WebCodecs does not even require that there is any format (format == null), in which case the frames can be drawn but copyTo() is not available.

The behavior you are seeing in Chrome is indeed due to how the hardware decoding path works. On Android the OS provides a GL texture which we can sample (as RGBA), rather than a buffer we can read in the native pixel format.

There is currently work being done to provide conversions (#92, #754), so if what you need is RGBA in CPU memory then WebCodecs should eventually provide that.