Smart Access Video Primer
The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions, and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. Any computer system has risks of security vulnerabilities that cannot be completely prevented or mitigated. AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of such revisions or changes. THIS INFORMATION IS PROVIDED ‘AS IS.” AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS, OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY RELIANCE, DIRECT, INDIRECT, SPECIAL, OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
AMD, the AMD Arrow logo, and combinations thereof are trademarks of Advanced Micro Devices, Inc. Other product names used in this publication are for identification purposes only and may be trademarks of their respective companies.
Video codec acceleration (including at least the HEVC (H.265), H.264, VP9, and AV1 codecs) is subject to and not operable without inclusion/installation of compatible media players. GD-176
© 2024 Advanced Micro Devices, Inc. All rights reserved.
© 2024 Advanced Micro Devices, Inc. All rights reserved.
Notice Regarding Standards. AMD does not provide a license or sublicense to any Intellectual Property Rights relating to any standards, including but not limited to any audio and/or video codec technologies such as MPEG-2, MPEG-4; AVC/H.264; HEVC/H.265; AV1; AAC decode/FFMPEG; AAC encode/FFMPEG; VC-1; and MP3 (collectively, the “Media Technologies”). For clarity, you will pay any royalties due for such third-party technologies, which may include the Media Technologies that are owed as a result of AMD providing the Software to you.
- 1 Outline
- 2 System Level Diagram
- 3 Case Study
- 4 Transcoding
- 5 Conclusion
- 6 Glossary of Acronyms
- 7 Annex: SAV Interfaces
We introduce SmartAccess Video (SAV) which parallelizes video decoding and encoding operations in systems with multiple instances of AMD’s hardware media coding engines, branded as Video Coding Next (VCN). These systems can take the form of an APU or a dGPU with multiple VCN instances; or a pairing of an APU plus one or more dGPUs, each with one or more VCNs. The total number of VCNs in a system influences the highest level of parallelization achieved. Next, we describe SAV optimization techniques and refer to the AMF interfaces needed for the SAV integration. As an illustration scenario, we provide the transcoding example and show what speed benefits SAV helps achieve.
Fig.1 : System Level Diagram
When SAV is enabled, AMF will take App processing commands of encode and decode and apply optimization logic to 1) increase parallelism of task execution, 2) adapt to system load and increase the rate of delivery, 3) synchronize across hardware components used for media processing, and 4) transfer resources across GPUs as needed. All these four operations happen at AMF and driver level, transparent to the calling App, and without a need to change current App calling flow. The requirement for the App to reap these benefits is to simply enable the SAV AMF property flags described in the Annex. In the absence of these flags, the App will select a GPU and then decode and encode streams on a single VCN within the selected GPU. If multiple VCNs exist on the system, it will miss the opportunities of parallelization.
We study the case of hardware-accelerated codec (encoding/decoding) conversion using the AMF SDK. We provide examples with the ‘TranscodeHW’ console app (similar to ffmpeg), along with optimization hints to third-party app developers.
Backgrounder: Video transcoding is the process of converting a video file from one format to another to ensure that it is compatible with various devices, networks, streaming/sharing platforms, and applications. It allows video to reach wider audiences. A major motivation behind transcoding is to ensure media file size reduction without losing the video’s quality. This occurs when converting legacy video formats to modern, more efficient standards. When converting to AV1, this also avoids costly licensing fees associated with HEVC.
Flow: Transcoding involves decoding the source video to raw images, optionally processing and modifying them, and then encoding the modified video content to a new format. AMF provides the means and necessary tools to optimize transcoding of video content for delivery, or for editing and post-production. The main challenge in these operations is the long processing time, and hence we will examine optimizing the timeline further.
Diagram
Fig. 2. Transcoding for delivery
SAV simplifies the computational challenge by empowering apps with parallelized decoding and encoding operations and therefore saves execution time. One possible parallelization scenario is illustrated below (parallelization is transparent to the App).
Fig. 3. Example encoding benefits when SAV enabled on an ‘AMD APU+AMD dGPU’ system
When SAV is enabled, cross-adapter transfer of the decoder target to the encoder is enabled by default, alleviating the calling App from handling it (see ‘Annex: SAV interfaces’ below for how to skip this transfer when unnecessary).
Fig. 4. Transcoding for editing
Transcoding for editing can support one or multiple streams to be decoded, enhanced, composited, and encoded into a final ‘Export.’ Sometimes the decoder targets are transferred back to the CPU. This creates opportunities for multiple parallelization scenarios. For example, the App can create multi decode sessions (optionally a mix of CPU and hardware parallel decoding sessions), then modify and submit the output to the AMF hardware encoder. With SAV enabled (using the decoder ‘Skip transfer’ property on the way, as no hardware decoder-hardware encoder communication is needed in this scenario, see ‘Annex: SAV Interfaces’), an optimization as to where hardware decoding and hardware encoding is allocated becomes implicit. SAV allows to increase the parallelization level among decoding and encoding and results in faster transcoding. Experiments demonstrated at event shows using OBS showed that enabling SAV reduced ‘skipped frames to encoding lag’ significantly in multi decode, single encode scene composition scenarios (frame drops improve from 80%+ in loads saturating a single VCN, down to 0% in the best cases when the load is spread using SAV).
SAV helps to implicitly accelerate muti-stream video processing such as transcoding (for editing or for delivery) by improving processing challenges through hardware parallelization on systems equipped with multiple AMD VCNs (such in an AMD APU and an AMD dGPU pairing). Through the AMF set of acceleration APIs for decoding and encoding, a video App can seamlessly take advantage of these acceleration benefits on applicable systems.
| Acronym | Definition |
|---|---|
| AMF | Advanced Media Framework, AMD’s media acceleration APIs |
| APU | Accelerated Processing Unit |
| dGPU | discrete GPU |
| SAV | SmartAccess Video |
| VCN | Video Coding Next (AMD’s codec hardware acceleration engine built into APUs and dGPUs) |
To enable SAV, an application must set the correct SAV property on the
AMFComponent prior to calling AMFComponent::Init. Different
AMFComponents have different SAV property names. The following shows the
property being set on each type of AMFComponent:
For the Decoder components "AMFVideoDecoder_*" (defined in VideoDecoderUVD.h):
pDecoder->SetProperty(AMF_VIDEO_DECODER_ENABLE_SMART_ACCESS_VIDEO, true);For the AVC Encoder component AMFVideoEncoderVCE_AVC,
AMFVideoEncoderVCE_SVC (defined in
VideoEncoderVCE.h )
pEncoderAvc->SetProperty(AMF_VIDEO_ENCODER_ENABLE_SMART_ACCESS_VIDEO, true);For the HEVC Encoder component AMFVideoEncoderHW_HEVC (defined
in VideoEncoderHEVC.h)
pEncoderHevc->SetProperty(AMF_VIDEO_ENCODER_HEVC_ENABLE_SMART_ACCESS_VIDEO, true);For the AV1 Encoder component AMFVideoEncoderHW_AV1 (defined in
VideoEncoderAV1.h)
pEncoderAv1->SetProperty(AMF_VIDEO_ENCODER_AV1_ENABLE_SMART_ACCESS_VIDEO, true);SAV enables an implicit transfer to move the output back to the original
AMFContext before it is returned from AMFComponent::QueryOutput (see
Fig. 2.).
For use cases where this transfer is not necessary (see Fig. 4), an
application can set the Decoder AMFComponent property
AMF_VIDEO_DECODER_SKIP_TRANSFER_SMART_ACCESS_VIDEO, leaving the
output wherever it was decoded. In this case, the following AMF APIs
that read to host memory can be safely called on the non-transferred
output from QueryOutput:
AMFData::Duplicate(AMF_MEMORY_HOST, ...)
AMFData::Convert(AMF_MEMORY_HOST)
AMFCompute::CopyPlaneToHost(...)
SAV is shown to end-users as an application decision. An application may allow the user to enable/disable it globally, per component, or set it on, completely hidden from the end user.
Optimization Hint: The use of ‘Low Latency’ properties.
When minimal latency or maximum performance is desired, an application
should set the ‘low latency’ property on the AMFComponent.
For the Decoder components "AMFVideoDecoder_" (defined in VideoDecoderUVD.h):
pDecoder->SetProperty(AMF_VIDEO_DECODER_LOW_LATENCY, true); For the AVC Encoder component AMFVideoEncoderVCE_AVC,
AMFVideoEncoderVCE_SVC (defined in
VideoEncoderVCE.h):
pEncoderAvc->SetProperty(AMF_VIDEO_ENCODER_LOWLATENCY_MODE, true); For the HEVC Encoder component AMFVideoEncoderVCE_HEVC,
AMFVideoEncoderVCE_SVC (defined in VideoEncoderHEVC.h)
pEncoderAvc->SetProperty(AMF_VIDEO_ENCODER_HEVC_LOWLATENCY_MODE, true); For the AV1 Encoder component AMFVideoEncoderHW_AV1 (defined in
VideoEncoderAV1.h)
pEncoderAv1->SetProperty(AMF_VIDEO_ENCODER_AV1_ENCODING_LATENCY_MODE,
AMF_VIDEO_ENCODER_AV1_ENCODING_LATENCY_MODE_LOWEST_LATENCY);