SmartDepthSync: Open Source Synchronized Video Recording System of Smartphone RGB and Depth Camera Range Image Frames with Sub-millisecond Precision

This is a repository that gathers open-source materials introduced and described in the SmartDepthSync paper (IEEE link, arXiv link). In addition to the paper, some description can be found in the links in the table below. The table keeps all the important link to the building parts of the system depicted in the image.

The materials represent diverse software and their own functionality launched on different hardware:

Material	Function	Launched on
Python launcher	command-line tool that is entry point for setup (launch of sensors, synchronization) and recording to ROS bag file procedures	mini-PC
ROS source code	driver for setting up and gathering data from the sensors (Azure Kinect DK depth camera and MCU board with IMU sensor)	mini-PC
MCU firmware	responsible for synchronized depth camera triggering and IMU data gathering	MCU board
OpenCamera-Sensors	smartphone application that records RGB video data and provides timestamps for synchronized depth camera triggering.	smartphone
bag-extractor	CLI-script that extracts recorded ROS bag files to images and other data types distributed to paths	any PC

For entire clone with submodules:

git clone --recurse-submodules https://github.com/MobileRoboticsSkoltech/bandeja-platform

Twist-n-Sync Python Package

Synchronization is based on our Twist-n-Sync algorithm that can be installed via pip.

Using this materials please do not forget to reference this repository and cite the paper:

@article{faizullin2022smartdepthsync,
  author={Faizullin, Marsel and Kornilova, Anastasiia and Akhmetyanov, Azat and Pakulev, Konstantin and Sadkov, Andrey and Ferrer, Gonzalo},
  title={SmartDepthSync: Open Source Synchronized Video Recording System of Smartphone RGB and Depth Camera Range Image Frames with Sub-millisecond Precision}, 
  journal={IEEE Sensors Journal}, 
  publisher={IEEE},
  year={2022},
  volume={22},
  number={7},
  pages={7043-7052},
  doi={10.1109/JSEN.2022.3150973}
}