MultiCam Visual Odometry

Design and Evaluation of a Generic Visual SLAM Framework for Multi Camera Systems

Version 0.2, April 10th, 2024

Authors: Pushyami Kaveti et al.
IEEE Robotics and Automation Letters (RA-L), 2023

Link: Paper | BibTex

1. Getting started

A. Prerequisites

We have tested this library in Ubuntu 16.04 and 20.04. The following external libraries are required for building Multicam Visual Odometry package. We have given the build/install instructions in the next section.

Dependencies:

• Opencv 3.3.1/4.5.5
• ROS Kinetic/Noetic
• Boost
• Eigen3
• GTSAM 4
• opengv
• gflags
• glog
• DBoW2
• DLib
• Pangolin
• python 2.7/3.8
• numpy
• YAML

[The entire package list can be built by writing a build.sh file.]

B. Build Instructions

1. ROS

Instructions to install ROS can be found in the links below:

• ROS Noetic
  • http://wiki.ros.org/noetic/Installation/Ubuntu
• ROS Kinetic
  • http://wiki.ros.org/kinetic/Installation/Ubuntu

2. Clone the repo

• Create a ROS catkin workspace

• Here on, we will assume that your catkin workspace location is ~/catkin_ws.

  cd ~/catkin_ws/src
  

  git clone https://github.com/neufieldrobotics/MultiCamSLAM
  

3. OpenCV

• Tested with OpenCV 3.3.1 and 4.5.5.
• For Ubuntu 20.04 + OpenCV 4.5.5 follow the below instructions.

  sudo apt update && sudo apt install -y cmake 
  
  mkdir ~/catkin_ws/ThirdParty && cd ~/catkin_ws/ThirdParty
  
  git clone https://github.com/opencv/opencv.git
  git clone https://github.com/opencv/opencv_contrib.git
  cd opencv
  git checkout 4.5.5
  cd ../opencv_contrib
  git checkout 4.5.5
  cd ../opencv
  mkdir build && cd build
  
  cmake -D CMAKE_BUILD_TYPE=RELEASE   -D CMAKE_INSTALL_PREFIX=../../opencv/install -D CMAKE_BUILD_TYPE=RELEASE  -D WITH_OPENGL=ON       -D OPENCV_EXTRA_MODULES_PATH=../../opencv_contrib/modules ..
  
  make install -j 4
  

4. Boost

• apt-get install cmake build-essential libboost-all-dev libgoogle-perftools-dev google-perftools  libatlas-base-dev libsuitesparse-dev libyaml-cpp-dev
  

5. Eigen3

• apt-get install wget unzip
  cd ~/catkin_ws/ThirdParty  
  wget https://gitlab.com/libeigen/eigen/-/archive/3.3.7/eigen-3.3.7.tar.gz
  tar zxf eigen-3.3.7.tar.gz
  mv eigen-3.3.7.tar.gz eigen
  

• In Eigen's CMakeLists.txt file, add the following below cmake_minimum_required

  add_compile_options(-std=c++17)
  

• cd ~/catkin_ws/ThirdParty/eigen
  cd eigen
  mkdir build && cd build
  cmake .. 
  sudo make install
  

6. GTSAM 4

• cd ~/catkin_ws/ThirdParty  
  wget https://github.com/borglab/gtsam/archive/refs/tags/4.1.1.zip 
  unzip 4.1.1.zip && rm 4.1.1.zip
  mv gtsam-4.1.1 gtsam
    
  cd gtsam
  mkdir build && cd build
  cmake .. -DCMAKE_INSTALL_PREFIX=../install
  make check
  make install
  
  

7. OpenGV

• cd ~/catkin_ws/ThirdParty 
  git clone https://github.com/laurentkneip/opengv
  

• In Opengv's CMakeLists.txt file, add the following below cmake_minimum_required

  add_compile_options(-std=c++17)
  

• cd opengv
  mkdir build && cd build
  cmake .. -DCMAKE_INSTALL_PREFIX=../install
  make install
  

• Run tests (Recommended)

  make test
  

8. gflags and glog

• This would have been installed as a part of the OpenCV build. If not, run this:

  • apt -y install libgoogle-glog-dev libgflags-dev
    

9. DBoW2

• cd ~/catkin_ws/ThirdParty 
  git clone https://github.com/PushyamiKaveti/DBoW2
  cd DBoW2
  mkdir build && cd build
  cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=../install
  make install
  

10. DLib

• Make sure you download the right one - there are 2 similar repositories - DLib and dlib

• cd ~/catkin_ws/ThirdParty 
  git clone https://github.com/dorian3d/DLib
  cd DLib
  mkdir build && cd build
  cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=../install
  make install
  

11. Pangolin

• Build instructions can be found in this link: https://github.com/stevenlovegrove/Pangolin

• cd ~/catkin_ws/ThirdParty 
  git clone --recursive https://github.com/stevenlovegrove/Pangolin.git
  cd Pangolin
  
  ./scripts/install_prerequisites.sh recommended
  cmake -B build
  cmake --build build
  
  ctest
  

C. Compile the ROS package

catkin_make

• catkin_make -DOpenCV_DIR=/home/$USER/catkin_ws/Third_party/opencv/build \
  -DDBoW2_DIR=/home/$USER/catkin_ws/Third_party/DBoW2/build \
  -DDLib_DIR=/home/$USER/catkin_ws/Third_party/DLib/build \
  -Dopengv_DIR=/home/$USER/catkin_ws/Third_party/opengv/build \
  -DGTSAM_DIR=/home/$USER/catkin_ws/Third_party/gtsam/build \
  -DGTSAM_UNSTABLE_DIR=/home/$USER/catkin_ws/Third_party/gtsam/build  \
  -Dopengv_INC_DIR=/home/$USER/catkin_ws/Third_party/opengv/include 
  

2. Running an example

a. Download the dataset

• Download the sample dataset from here.

b. Setup the config files

• edit the following paramters in src/MultiCamSLAM/MCApps/params/lf_frontend.yaml

  • LogDir - Provide the path to save the log files
    • Example: ~/catkin_ws/src/MultiCamSLAM/log/
  • Vocabulary - Provide the path to the ORB Vocabulary files
    • Example: ~/catkin_ws/src/MultiCamSLAM/MCApps/params/ORBvoc.txt

• edit the following parameters in src/MultiCamSLAM/MCApps/params/lf_slam_config.cfg

  • data_path - Provide the path to the downloaded dataset
    • Example: /home/marley/catkin_ws/ISEC_Lab1/
  • calib_file_path - Provide the path to the settings file for a particular multi-camera rig.
    • Example: /home/marley/catkin_ws/ISEC_Lab1/calib/02_23_2022_5cams_camchain.yaml
  • images_path - Provide the path to the images folder in the downloaded dataset
    • Example: /home/marley/catkin_ws/ISEC_Lab1/image_data/
  • frontend_params_file - Provide the complete path to the lf_frontend.yaml file in MCApps/params of the package.
    • Example: /home/marley/catkin_ws/src/MultiCamSLAM/MCApps/params/lf_frontend.yaml
  • backend_params_file - Provide the complete path to the lf_backend.yaml file in MCApps/params of the package.
    • Example: /home/marley/catkin_ws/src/MultiCamSLAM/MCApps/params/lf_backend.yaml

c. Run

In Terminal 1

roscore

In Terminal 2 Edit the below command based on the path to the cfg file.

./devel/lib/MCApps/MCSlamapp --config_file /home/marley/neu_ws/src/MultiCamSLAM/MCApps/params/lf_slam_config.cfg --log_file /home/marley/log.txt --traj_file /home/marley/traj.txt

3. Additonal Details from the paper

a. Setup

The custom-built multi-camera rig used to collect data for evaluating the SLAM pipeline.

b. Qualitative Results

i. Curry Center Dataset

Estimated trajectories of the Curry center sequence with outdoor data and dynamic content. Stars indicate final positions of trajectory estimates. Accuracy and robustness improve with increasing number of cameras in OV configurations, as shown by accumulated drift in final position. Red and blue boxes highlight tracking failures caused by occluding dynamic objects. N-OV configuration exhibits scale issues compared to OV configuration but is robust to dynamic content

ii. ISEC_Ground1 Dataset

Estimated trajectories of the ISEC_Ground1 sequence. Here, the robot’s start and end positions are the same, facilitating performance evaluation. We achieve comparable results to ORBSLAM3 and SVO in stereo setup and demonstrate improved accuracy with increasing overlapping cameras.

iii. ISEC_Lab1 Dataset

Estimated trajectories of the ISEC_Lab1 sequence. Here, the ground truth is shown as a dashed line. We achieve comparable results to ORBSLAM3 and SVO in stereo setup and demonstrate improved accuracy with increasing overlapping cameras.

Citation

If you use this work in an academic context, please cite the following publication:

@ARTICLE{10253964,

  author={Kaveti, Pushyami and Vaidyanathan, Shankara Narayanan and Chelvan, Arvind Thamil and Singh, Hanumant},

  journal={IEEE Robotics and Automation Letters}, 

  title={Design and Evaluation of a Generic Visual SLAM Framework for Multi Camera Systems}, 

  year={2023},

  volume={8},

  number={11},

  pages={7368-7375},

  doi={10.1109/LRA.2023.3316609}}