AMD GPU support with the rocm detector and YOLOv8 pretrained model do…

…wnload (#9762)

* ROCm AMD/GPU based build and detector, WIP

* detectors/rocm: separate yolov8 postprocessing into own function; fix box scaling; use cv2.dnn.blobForImage for preprocessing; assert on required model parameters

* AMD/ROCm: add couple of more ultralytics models; comments

* docker/rocm: make imported model files readable by all

* docker/rocm: readme about running on AMD GPUs

* docker/rocm: updated README

* docker/rocm: updated README

* docker/rocm: updated README

* detectors/rocm: separated preprocessing functions into yolo_utils.py

* detector/plugins: added onnx cpu plugin

* docker/rocm: updated container with limite label sets

* example detectors view

* docker/rocm: updated README.md

* docker/rocm: update README.md

* docker/rocm: do not set HSA_OVERRIDE_GFX_VERSION at all for the general version as the empty value broke rocm

* detectors: simplified/optimized yolov8_postprocess

* detector/yolo_utils: indentation, remove unused variable

* detectors/rocm: default option to conserve cpu usage at the expense of latency

* detectors/yolo_utils: use nms to prefilter overlapping boxes if too many detected

* detectors/edgetpu_tfl: add support for yolov8

* util/download_models: script to download yolov8 model files

* docker/main: add download-models overlay into s6 startup

* detectors/rocm: assume models are in /config/model_cache/yolov8/

* docker/rocm: compile onnx files into mxr files at startup

* switch model download into bash script

* detectors/rocm: automatically override HSA_OVERRIDE_GFX_VERSION for couple of known chipsets

* docs: rocm detector first notes

* typos

* describe builds (harakas temporary)

* docker/rocm: also build a version for gfx1100

* docker/rocm: use cp instead of tar

* docker.rocm: remove README as it is now in detector config

* frigate/detectors: renamed yolov8_preprocess->preprocess, pass input tensor element type

* docker/main: use newer openvino (2023.3.0)

* detectors: implement class aggregation

* update yolov8 model

* add openvino/yolov8 support for label aggregation

* docker: remove pointless s6/timeout-up files

* Revert "detectors: implement class aggregation"

This reverts commit dcfe6bb.

* detectors/openvino: remove class aggregation

* detectors: increase yolov8 postprocessing score trershold to 0.5

* docker/rocm: separate rocm distributed files into its own build stage

* Update object_detectors.md

* updated CODEOWNERS file for rocm

* updated build names for documentation

* Revert "docker/main: use newer openvino (2023.3.0)"

This reverts commit dee95de.

* reverrted openvino detector

* reverted edgetpu detector

* scratched rocm docs from any mention of edgetpu or openvino

* Update docs/docs/configuration/object_detectors.md

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

* renamed frigate.detectors.yolo_utils.py -> frigate.detectors.util.py

* clarified rocm example performance

* Improved wording and clarified text

* Mentioned rocm detector for AMD GPUs

* applied ruff formating

* applied ruff suggested fixes

* docker/rocm: fix missing argument resulting in larger docker image sizes

* docs/configuration/object_detectors: fix links to yolov8 release files

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

CODEOWNERS

@@ -2,5 +2,5 @@
 /docker/tensorrt/ @madsciencetist @NateMeyer
 /docker/tensorrt/*arm64* @madsciencetist
 /docker/tensorrt/*jetson* @madsciencetist
-
 /docker/rockchip/ @MarcA711
+/docker/rocm/ @harakas

docker/main/Dockerfile

@@ -196,6 +196,8 @@ EXPOSE 8555/tcp 8555/udp
 
 # Configure logging to prepend timestamps, log to stdout, keep 0 archives and rotate on 10MB
 ENV S6_LOGGING_SCRIPT="T 1 n0 s10000000 T"
+# Do not fail on long-running download scripts
+ENV S6_CMD_WAIT_FOR_SERVICES_MAXTIME=0
 
 ENTRYPOINT ["/init"]
 CMD []

docker/main/requirements-wheels.txt

@@ -25,6 +25,7 @@ norfair == 2.2.*
 setproctitle == 1.3.*
 ws4py == 0.5.*
 unidecode == 1.3.*
+onnxruntime == 1.16.*
 # Openvino Library - Custom built with MYRIAD support
 openvino @ https://github.com/NateMeyer/openvino-wheels/releases/download/multi-arch_2022.3.1/openvino-2022.3.1-1-cp39-cp39-manylinux_2_31_x86_64.whl; platform_machine == 'x86_64'
 openvino @ https://github.com/NateMeyer/openvino-wheels/releases/download/multi-arch_2022.3.1/openvino-2022.3.1-1-cp39-cp39-linux_aarch64.whl; platform_machine == 'aarch64'

docker/main/rootfs/etc/s6-overlay/s6-rc.d/download-models/run

@@ -0,0 +1,34 @@
+#!/command/with-contenv bash
+# shellcheck shell=bash
+# Download yolov8 models when DOWNLOAD_YOLOV8=1 environment variable is set
+
+set -o errexit -o nounset -o pipefail
+
+MODEL_CACHE_DIR=${MODEL_CACHE_DIR:-"/config/model_cache"}
+YOLOV8_DIR="$MODEL_CACHE_DIR/yolov8"
+YOLOV8_URL=https://github.com/harakas/models/releases/download/yolov8.1-1.1/yolov8.small.models.tar.gz
+YOLOV8_DIGEST=304186b299560fbacc28eac9b9ea02cc2289fe30eb2c0df30109a2529423695c
+
+if [ "$DOWNLOAD_YOLOV8" = "1" ]; then
+  echo "download-models: DOWNLOAD_YOLOV8=${DOWNLOAD_YOLOV8}, running download"
+  if ! test -f "${YOLOV8_DIR}/model.fetched"; then
+    mkdir -p $YOLOV8_DIR
+    TMP_FILE="${YOLOV8_DIR}/download.tar.gz"
+    curl --no-progress-meter -L --max-filesize 500M --insecure --output $TMP_FILE "${YOLOV8_URL}"
+    digest=$(sha256sum $TMP_FILE | awk '{print $1}')
+    if [ "$digest" = "$YOLOV8_DIGEST" ]; then
+      echo "download-models: Extracting downloaded file"
+      cd $YOLOV8_DIR
+      tar zxf $TMP_FILE
+      rm $TMP_FILE
+      touch model.fetched
+      echo "download-models: Yolov8 download done, files placed into ${YOLOV8_DIR}"
+    else
+      echo "download-models: Downloaded file digest does not match: got $digest, expected $YOLOV8_DIGEST"
+      rm $TMP_FILE
+    fi
+  else
+    echo "download-models: ${YOLOV8_DIR}/model.fetched already present"
+  fi
+fi
+

docker/main/rootfs/etc/s6-overlay/s6-rc.d/download-models/type

@@ -0,0 +1 @@
+oneshot

docker/main/rootfs/etc/s6-overlay/s6-rc.d/download-models/up

@@ -0,0 +1 @@
+/etc/s6-overlay/s6-rc.d/download-models/run

docker/rocm/Dockerfile

@@ -0,0 +1,106 @@
+# syntax=docker/dockerfile:1.4
+
+# https://askubuntu.com/questions/972516/debian-frontend-environment-variable
+ARG DEBIAN_FRONTEND=noninteractive
+ARG ROCM=5.7.3
+ARG AMDGPU=gfx900
+ARG HSA_OVERRIDE_GFX_VERSION
+ARG HSA_OVERRIDE
+
+#######################################################################
+FROM ubuntu:focal as rocm
+
+ARG ROCM
+
+RUN apt-get update && apt-get -y upgrade
+RUN apt-get -y install gnupg wget
+
+RUN mkdir --parents --mode=0755 /etc/apt/keyrings
+
+RUN wget https://repo.radeon.com/rocm/rocm.gpg.key -O - | gpg --dearmor | tee /etc/apt/keyrings/rocm.gpg > /dev/null
+COPY docker/rocm/rocm.list /etc/apt/sources.list.d/
+COPY docker/rocm/rocm-pin-600 /etc/apt/preferences.d/
+
+RUN apt-get update
+
+RUN apt-get -y install --no-install-recommends migraphx
+RUN apt-get -y install --no-install-recommends migraphx-dev
+
+RUN mkdir -p /opt/rocm-dist/opt/rocm-$ROCM/lib
+RUN cd /opt/rocm-$ROCM/lib && cp -dpr libMIOpen*.so* libamd*.so* libhip*.so* libhsa*.so* libmigraphx*.so* librocm*.so* librocblas*.so* /opt/rocm-dist/opt/rocm-$ROCM/lib/
+RUN cd /opt/rocm-dist/opt/ && ln -s rocm-$ROCM rocm
+
+RUN mkdir -p /opt/rocm-dist/etc/ld.so.conf.d/
+RUN echo /opt/rocm/lib|tee /opt/rocm-dist/etc/ld.so.conf.d/rocm.conf
+
+#######################################################################
+FROM --platform=linux/amd64 debian:11 as debian-base
+
+RUN apt-get update && apt-get -y upgrade
+RUN apt-get -y install --no-install-recommends libelf1 libdrm2 libdrm-amdgpu1 libnuma1 kmod
+
+RUN apt-get -y install python3
+
+#######################################################################
+# ROCm does not come with migraphx wrappers for python 3.9, so we build it here
+FROM debian-base as debian-build
+
+ARG ROCM
+
+COPY --from=rocm /opt/rocm-$ROCM /opt/rocm-$ROCM
+RUN ln -s /opt/rocm-$ROCM /opt/rocm
+
+RUN apt-get -y install g++ cmake
+RUN apt-get -y install python3-pybind11 python3.9-distutils python3-dev
+
+WORKDIR /opt/build
+
+COPY docker/rocm/migraphx .
+
+RUN mkdir build && cd build && cmake .. && make install
+
+#######################################################################
+FROM deps AS deps-prelim
+
+# need this to install libnuma1
+RUN apt-get update
+# no ugprade?!?!
+RUN apt-get -y install libnuma1
+
+WORKDIR /opt/frigate/
+COPY --from=rootfs / /
+COPY docker/rocm/rootfs/ /
+
+#######################################################################
+FROM scratch AS rocm-dist
+
+ARG ROCM
+ARG AMDGPU
+
+COPY --from=rocm /opt/rocm-$ROCM/bin/rocminfo /opt/rocm-$ROCM/bin/migraphx-driver /opt/rocm-$ROCM/bin/
+COPY --from=rocm /opt/rocm-$ROCM/share/miopen/db/*$AMDGPU* /opt/rocm-$ROCM/share/miopen/db/
+COPY --from=rocm /opt/rocm-$ROCM/lib/rocblas/library/*$AMDGPU* /opt/rocm-$ROCM/lib/rocblas/library/
+COPY --from=rocm /opt/rocm-dist/ /
+COPY --from=debian-build /opt/rocm/lib/migraphx.cpython-39-x86_64-linux-gnu.so /opt/rocm-$ROCM/lib/
+
+#######################################################################
+FROM deps-prelim AS rocm-prelim-hsa-override0
+
+ENV HSA_ENABLE_SDMA=0
+
+COPY --from=rocm-dist / /
+
+RUN ldconfig
+
+#######################################################################
+FROM rocm-prelim-hsa-override0 as rocm-prelim-hsa-override1
+
+ARG HSA_OVERRIDE_GFX_VERSION
+ENV HSA_OVERRIDE_GFX_VERSION=$HSA_OVERRIDE_GFX_VERSION
+
+#######################################################################
+FROM rocm-prelim-hsa-override$HSA_OVERRIDE as rocm-deps
+
+# Request yolov8 download at startup
+ENV DOWNLOAD_YOLOV8=1
+

docker/rocm/migraphx/CMakeLists.txt

@@ -0,0 +1,26 @@
+
+cmake_minimum_required(VERSION 3.1)
+
+set(CMAKE_CXX_STANDARD 17)
+set(CMAKE_CXX_STANDARD_REQUIRED ON)
+set(CMAKE_CXX_EXTENSIONS OFF)
+
+if(NOT CMAKE_BUILD_TYPE)
+  set(CMAKE_BUILD_TYPE Release)
+endif()
+
+SET(CMAKE_INSTALL_RPATH_USE_LINK_PATH TRUE)
+
+project(migraphx_py)
+
+include_directories(/opt/rocm/include)
+
+find_package(pybind11 REQUIRED)
+pybind11_add_module(migraphx migraphx_py.cpp)
+
+target_link_libraries(migraphx PRIVATE /opt/rocm/lib/libmigraphx.so /opt/rocm/lib/libmigraphx_tf.so /opt/rocm/lib/libmigraphx_onnx.so)
+
+install(TARGETS migraphx
+  COMPONENT python
+  LIBRARY DESTINATION /opt/rocm/lib
+)