Metadata overlays with different aspect ratios between Video Source and Video Encoder.

[jcbeaulieu]

When the aspect ratio of the encoded stream is different from the sensor's aspect ratio or when the sensor image is cropped in any other way, the given (video and metadata) streams have no embedded information that can allow a video player to know where the encoded image is present in relation to the sensor.

This can occur for:

• On device dewarping
• On device digital zooming
• When changing cameras on a video encoder.
• When using an encoding resolution with a different aspect ratio than the video sensor resolution.

When a video player tries to overlay the received stream, it can cause bounding boxes and other objects to become misaligned, with no information in the metadata stream itself to correct for the distortion.

[sujithhanwha]

Related to this issue #410

[jcbeaulieu]

Concerning the answer to #410 , if the bounds defined in the video source configuration is necessary, then would this mean that it's impossible to have correct overlay position for on-device digital zoom since the bounds are always changing?
Would it also make it impossible for a video player to place the metadata overlay correctly using only the video and metadata streams, requiring a VMS to store the bounds information in addition to the metadata stream and the video stream?