jetson docker image (#119)

* nvidia docker image builds also moved x86 dockerfile to docker folder. also also obcpp segfaults when run in the container. will look into soon * cuda_check integration test * docker dir makefile * build all binaries also add a comment for where base image came from * fix build_jetson_image paths * cuda_check builds and works * nvidia docker image builds also moved x86 dockerfile to docker folder. also also obcpp segfaults when run in the container. will look into soon * cuda_check integration test * docker dir makefile * build all binaries also add a comment for where base image came from * fix build_jetson_image paths * cuda_check builds and works * github action to build jetson dockerfile * pull_request * github actions might work? * jetson Docker: compile obcpp and cuda_check * bespoke github env * trying to free github runner space * disable half operators? * trying dusty-nv's torchvision install * moved back to run make for torchvision * jetson docker composes * disable jetson docker build not working due to an error compiling half operators in CUDA programs on github actions torch/cutorch#797 https://github.com/tritonuas/obcpp/actions/runs/8637979133/job/23681389912#step:8:6390 * devcontianer use new x86 tag * remove Dockerfile.nvidia in favor of Dockerfile.jetson * instructions for building jetson image --------- Co-authored-by: tuas-travis-ci <ucsdtuas@gmail.com>
tritonuas · Apr 10, 2024 · 10b0273 · 10b0273
1 parent 1ed9253
commit 10b0273
Show file tree

Hide file tree

Showing 13 changed files with 423 additions and 8 deletions.
diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json
@@ -1,7 +1,7 @@
 // See this page for reference of options: https://containers.dev/implementors/json_reference
 {
 	"name": "Existing Dockerfile",
-	"image": "ghcr.io/tritonuas/obcpp:main",
+	"image": "ghcr.io/tritonuas/obcpp:x86",
 	// enable when need to connect over USB to pixhawk
 	// also: need to run obcpp with sudo or add tuas user to dialout group with
 	// 	   `sudo usermod -aG dialout tuas && newgrp && bash`

diff --git a/.github/workflows/docker-jetson.yml b/.github/workflows/docker-jetson.yml
@@ -0,0 +1,82 @@
+# name: Push Jetson Docker Image
+#
+# on:
+#   push:
+#     branches:
+#       - main
+#   pull_request:
+#
+# env:
+#   REGISTRY: ghcr.io
+#   IMAGE_NAME: ${{ github.repository }}
+#   TAG: "jetson"
+#   DOCKERFILE_PATH: 'docker/Dockerfile.jetson'
+#
+# jobs:
+#   changes:
+#     runs-on: ubuntu-latest
+#     outputs:
+#       dockerfile: ${{ steps.changes.outputs.dockerfile }}
+#     steps:
+#     - uses: actions/checkout@v3
+#     - uses: dorny/paths-filter@v2
+#       id: changes
+#       with:
+#         filters: |
+#           dockerfile:
+#             - ${{ env.DOCKERFILE_PATH }}
+#
+#   build-and-push-image:
+#     needs: changes
+#     if: ${{ needs.changes.outputs.dockerfile == 'true' }}
+#     runs-on: ubuntu-latest
+#     permissions:
+#       contents: read
+#       packages: write
+#     steps:
+#       - name: Free Disk Space (Ubuntu)
+#         uses: jlumbroso/free-disk-space@main
+#         with:
+#           # this might remove tools that are actually needed,
+#           # if set to "true" but frees about 6 GB
+#           tool-cache: true
+#           
+#           # all of these default to true, but feel free to set to
+#           # "false" if necessary for your workflow
+#           android: true
+#           dotnet: true
+#           haskell: true
+#           large-packages: true
+#           swap-storage: true
+#       - name: Checkout repository
+#         uses: actions/checkout@v4
+#
+#       - name: Set up QEMU
+#         uses: docker/setup-qemu-action@v2
+#     
+#       - name: Set up Docker Buildx
+#         uses: docker/setup-buildx-action@v2
+#
+#       - name: Log in to the Container registry
+#         uses: docker/login-action@65b78e6e13532edd9afa3aa52ac7964289d1a9c1
+#         with:
+#           registry: ${{ env.REGISTRY }}
+#           username: ${{ github.actor }}
+#           password: ${{ secrets.GITHUB_TOKEN }}
+#
+#       - name: Extract metadata (tags, labels) for Docker
+#         id: meta
+#         uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7
+#         with:
+#           images: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+#
+#       - name: Build and push Docker image
+#         uses: docker/build-push-action@f2a1d5e99d037542a71f64918e516c093c6f3fc4
+#         with:
+#           platforms: linux/arm64
+#           file: ${{ env.DOCKERFILE_PATH }} 
+#           context: .
+#           push: true
+#           tags: ${{ env.TAG }} 
+#           labels: ${{ steps.meta.outputs.labels }}
+#
diff --git a/.github/workflows/docker.yml → .github/workflows/docker-x86.yml b/.github/workflows/docker.yml → .github/workflows/docker-x86.yml
@@ -1,4 +1,4 @@
-name: Push Docker Image
+name: Push x86 Docker Image
 
 on:
   push:
@@ -8,6 +8,8 @@ on:
 env:
   REGISTRY: ghcr.io
   IMAGE_NAME: ${{ github.repository }}
+  TAG: "x86"
+  DOCKERFILE_PATH: 'docker/Dockerfile.x86'
 
 jobs:
   changes:
@@ -21,7 +23,7 @@ jobs:
       with:
         filters: |
           dockerfile:
-            - '.devcontainer/Dockerfile'
+            - ${{ env.DOCKERFILE_PATH }}
 
   build-and-push-image:
     needs: changes
@@ -50,9 +52,9 @@ jobs:
       - name: Build and push Docker image
         uses: docker/build-push-action@f2a1d5e99d037542a71f64918e516c093c6f3fc4
         with:
-          file: .devcontainer/Dockerfile
+          file: ${{ env.DOCKERFILE_PATH }} 
           context: .
           push: true
-          tags: ${{ steps.meta.outputs.tags }}
+          tags: ${{ env.TAG }} 
           labels: ${{ steps.meta.outputs.labels }}
 
diff --git a/.gitignore b/.gitignore
@@ -3,7 +3,6 @@ lib/
 bin/
 logs/
 
-Makefile
 CMakeFiles/
 build/
 CMakeCache.txt
@@ -26,4 +25,4 @@ libcore_library.dylib
 .DS_Store
 
 **/*.pb.cc
-**/*.pb.h
+**/*.pb.h
diff --git a/CMakeLists.txt b/CMakeLists.txt
@@ -112,6 +112,20 @@ add_subdirectory(src)
 
 add_subdirectory(tests/integration)
 
+# cuda_check
+add_executable(cuda_check ${SOURCES} tests/integration/cuda_check.cpp)
+target_add_torch(cuda_check)
+target_add_json(cuda_check)
+target_add_httplib(cuda_check)
+target_add_mavsdk(cuda_check)
+target_add_matplot(cuda_check)
+target_add_protobuf(cuda_check)
+target_add_opencv(cuda_check) 
+target_add_loguru(cuda_check)
+# for some reason calling target_add_imagemagick here conflicts with, so we are including/linking without the function call
+# target_add_imagemagick(cuda_check)
+target_include_directories(cuda_check PRIVATE ${ImageMagick_INCLUDE_DIRS})
+target_link_libraries(cuda_check PRIVATE -Wl,--copy-dt-needed-entries ${ImageMagick_LIBRARIES})
 # =============================
 
 # =============================

diff --git a/docker/Dockerfile.arm b/docker/Dockerfile.arm
@@ -0,0 +1,133 @@
+FROM arm64/ubuntu:22.04
+
+ARG USERNAME=tuas USER_UID=1000 USER_GID=1000 DEBIAN_FRONTEND=noninteractive
+
+# Needed to spawn up a GUI in headless mode for matplotplus to work
+ENV QT_QPA_PLATFORM="vnc"
+
+# Create a non-root user
+RUN groupadd -f --gid $USER_GID $USERNAME \
+    && useradd --uid $USER_UID --gid $USER_GID -m $USERNAME
+# exit 0 ensures that it won't prematurely stop if for some reason the user already exists
+
+# https://gist.github.com/SSARCandy/fc960d8905330ac695e71e3f3807ce3d
+# OpenCV dependencies from above
+RUN --mount=target=/var/lib/apt/lists,type=cache,sharing=locked \
+    --mount=target=/var/cache/apt,type=cache,sharing=locked \
+    rm -f /etc/apt/apt.conf.d/docker-clean \ 
+    && apt-get update \
+    && apt-get upgrade -y \
+    && apt-get install -y build-essential \
+                           software-properties-common \
+                           sudo \
+                           gdb \
+                           git \
+                           wget \
+                           ccache \
+                           vim \
+                           curl \
+                           unzip \
+                           protobuf-compiler \
+                           libopencv-dev \
+                           # Need these to install Python 3.11 from pyenv
+                           python3-pip \
+                           libssl-dev \
+                           zlib1g-dev \
+                           libbz2-dev \
+                           libreadline-dev \
+                           libsqlite3-dev \
+                           libncursesw5-dev \
+                           xz-utils \
+                           tk-dev \
+                        #    libxm12-dev \
+                           libxmlsec1-dev \
+                           libffi-dev \
+                           liblzma-dev \
+                           # needed for matplotplus
+                           gnuplot \ 
+                           # imagemagick with c++ dependency
+                           graphicsmagick-libmagick-dev-compat 
+
+RUN apt-add-repository universe
+RUN apt-get install -y cpplint
+
+# Update Python to 3.11 so that libtorch can be properly built from source
+ENV PYENV_ROOT="/.pyenv"
+RUN curl https://pyenv.run | bash
+ENV PATH="${PATH}:${PYENV_ROOT}/bin"
+RUN eval "$(pyenv init -)"
+RUN pyenv install 3.11
+RUN pyenv global 3.11
+RUN pip3 install typing-extensions PyYAML
+
+RUN echo $USERNAME ALL=\(root\) NOPASSWD:ALL > /etc/sudoers.d/$USERNAME \
+    && chmod 0440 /etc/sudoers.d/$USERNAME
+
+# Download latest CMake from their repositories
+RUN --mount=target=/var/lib/apt/lists,type=cache,sharing=locked \
+    --mount=target=/var/cache/apt,type=cache,sharing=locked \
+    rm -f /etc/apt/apt.conf.d/docker-clean \ 
+    && apt-get update \
+    && wget https://github.com/Kitware/CMake/releases/download/v3.27.7/cmake-3.27.7-linux-x86_64.sh \
+      -q -O /tmp/cmake-install.sh \
+      && chmod u+x /tmp/cmake-install.sh \
+      && mkdir /opt/cmake-3.24.1 \
+      && /tmp/cmake-install.sh --skip-license --prefix=/opt/cmake-3.24.1 \
+      && rm /tmp/cmake-install.sh \
+      && ln -s /opt/cmake-3.24.1/bin/* /usr/local/bin
+
+# install prebuilt MAVSDK to system
+# https://github.com/mavlink/MAVSDK/releases
+ARG MAVSDK_VERSION=1.4.17
+ARG MAVSDK_DEB=libmavsdk-dev_${MAVSDK_VERSION}_debian11_arm64.deb
+RUN wget https://github.com/mavlink/MAVSDK/releases/download/v${MAVSDK_VERSION}/${MAVSDK_DEB} \
+    && dpkg -i ${MAVSDK_DEB} \
+    && rm ${MAVSDK_DEB}
+
+# the official docs say also these
+# https://docs.opencv.org/4.x/d7/d9f/tutorial_linux_install.html
+# cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
+# python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
+
+# pull pre-built libtorch
+ARG LIBTORCH_VERSION=2.1.0
+ARG LIBTORCH_INSTALL_DIR=/libtorch-tmp
+WORKDIR ${LIBTORCH_INSTALL_DIR}
+RUN git clone -b main --recurse-submodule https://github.com/pytorch/pytorch.git
+RUN mkdir pytorch-build
+RUN cd pytorch-build
+RUN cmake -D_GLIBCXX_USE_CXX11_ABI=1 -DBUILD_SHARED_LIBS:BOOL=ON -DCMAKE_BUILD_TYPE:STRING=Release -DPYTHON_EXECUTABLE:PATH=`which python3` -DCMAKE_PREFIX_PATH=../pytorch-install ${LIBTORCH_INSTALL_DIR}/pytorch
+RUN cmake --build . --target install
+
+# RUN wget "https://download.pytorch.org/libtorch/cpu/libtorch-cxx11-abi-shared-with-deps-${LIBTORCH_VERSION}%2Bcpu.zip" \
+#     && unzip "libtorch-cxx11-abi-shared-with-deps-${LIBTORCH_VERSION}+cpu.zip"
+# needed to build torchvision below and to build targets that use libtorch inside the container
+ENV CMAKE_PREFIX_PATH="${LIBTORCH_INSTALL_DIR}/libtorch"
+
+# pull and build torchvision
+# refer to this page for version compatibilty with pytorch (libtorch) https://github.com/pytorch/pytorch/wiki/PyTorch-Versions
+ARG TORCHVISION_VERSION=0.16.0
+ARG TORCHVISION_INSTALL_DIR=/torchvision-tmp
+WORKDIR ${TORCHVISION_INSTALL_DIR} 
+RUN wget "https://github.com/pytorch/vision/archive/refs/tags/v${TORCHVISION_VERSION}.zip" \
+    && unzip "v${TORCHVISION_VERSION}.zip" \
+    && mkdir -p "${TORCHVISION_INSTALL_DIR}/vision-${TORCHVISION_VERSION}/build" \
+    && cd "${TORCHVISION_INSTALL_DIR}/vision-${TORCHVISION_VERSION}/build" \
+    && cmake -DWITH_CUDA=off -D_GLIBCXX_USE_CXX11_ABI=1 -DCMAKE_BUILD_TYPE=Release .. \
+    && make -j2 \
+    && make install
+
+WORKDIR /obcpp
+COPY . .
+
+RUN rm -rf /obcpp/build
+WORKDIR /obcpp/build
+ENV CMAKE_PREFIX_PATH="/usr/local/lib/python3.8/dist-packages/torch/share/cmake/Torch;/usr/local/share/cmake/TorchVision"
+RUN GITHUB_ACTIONS=true cmake -DCMAKE_PREFIX_PATH="/usr/local/lib/python3.8/dist-packages/torch/share/cmake/Torch;/usr/local/share/cmake/TorchVision" -DCMAKE_MODULE_PATH="/usr/local/share/cmake/TorchVision" -DCMAKE_BUILD_TYPE="Debug" ..
+
+RUN make cuda_check VERBOSE=1
+
+# login as non-root user
+# USER $USERNAME
+
+CMD [ "/obcpp/build/bin/obcpp" ]