Benchmarking Jetson Edge Devices with an End-to-end Video-based Anomaly Detection System

This repo contains the Pytorch implementation of our paper:

Benchmarking Jetson Edge Devices with an End-to-end Video-based Anomaly Detection System

Hoang V Pham, Thinh G Tran, Chuong D Le, An D Le, Hien B Vo

• Accepted at FICC 2024.
• Acknowledgment: Grateful for Mr. Dat Thanh Vo, from University of Windsor, Canada, for the support on the process of designing a 3D cover box. Grateful for Mr. Anh Duy Pham, from Hochschule Bonn-Rhein-Sieg Sankt Augustin, Germany for the constructive comments during the deployment process.

This repository showcases an end-to-end video-based anomaly detection that is capable of running in 🔥 real-time 🔥 on NVIDIA Jetson Orin Platforms with NVIDIA TensorRT. This repository is built based on RTFM as the anomaly detector model and SlowFast as for the end-to-end codebase from camera (and video) directly.

🤸 Examples

Example - Detect a video labeled as "arson" from UCF-Crime testset

The input is a random video labeled as "arson" from UCF-Crime testset. The input path is specified in the config file.

🛠️ Prerequisites

Please download the Resnet50 I3D features extractor and RTFM models checkpoints training on UCF-Crime dataset from links below and save the ckpt folder in the current repository:

I3D non-local pretrain feature extractor

RTFM training on UCF-Crime dataset

Please download videos data from HERE & save in data folder for testing

⏱️ Performance

End-to-end system evaluation on multiple Jetson Edge devices

	Jeson Nano (4GB)		Jetson AGX Xavier (32GB)		Jetson Orin Nano (8GB)
	Without Torch-TensorRT	Torch-TensorRT	Without Torch-TensorRT	Torch-TensorRT	Without Torch-TensorRT	Torch-TensorRT
RAM usage (GB)	X	2.61	5.72	3.74	4.94	3.11
Frame per Second (FPS)	X	1.55	29.57	41.65	36.02	47.56

Notes

In this testing environment, all devices shared the same setup parameters. The system needs 3.11 GB RAM for active operation, in contrast to the Jetson AGX Xavier’s 3.74 GB consumption, denoting a more RAM-efficient usage. The system is only operational on Jetson Nano with Torch-TensorRT because of the limited RAM resource compared to the required RAM usage from the system. The Jetson Nano with Torch-TensorRT displayed minimal RAM consumption at 2.61 GB while having the slowest speed, 1.55 FPS only.

Contrastingly, the Jetson Orin Nano proved the fastest speed with 47.56 FPS, amounting to almost 30 times the speed of the Jetson Nano, additionally surpassing the Jetson AGX Xavier by nearly 15% under identical setup conditions. During anomaly detection system operation, the Jetson Orin Nano exhibited half the power consumption compared to Jetson AGX Xavier. As such, the Jetson Orin Nano 8 GB RAM with Torch-TensorRT emerged as the most effective device for anomaly detection system implementation, surpassing the other compared options in terms of efficiency.

🛠️ Setup

To get starting with the repository.

1. Install the dependencies

   1. Install conda environment with Python3.10

   2. Install dependencies

      pip install -r requirements.txt

   3. Install psycopg2

      conda install -c conda-forge psycopg2

   4. Install detectron2 dependencies

      pip install 'git+https://github.com/facebookresearch/detectron2.git'

2. Configuration

   All information about system parameters are in configuration file configs/ucf_RTFM_I3D_slowfast.yaml, including input & output video path.

3. Run the basic usage example

   1. Perform the dection from an input video demo

      python -m main --cfg configs/ucf_RTFM_I3D_slowfast.yaml
      

      This outputs a result to output/[video_input_name].avi

   2. Perform the dection directly from camera

      In the config file above, please change WEBCAM parameter to -1 and comment the OUTPUT_FILE parameter

4. Deploy the repository on Docker (optional)

   Please open Dockerfile file to get the deployment script & instruction to begin after deployment on Jetson Orin Nano

   Files py2trt_i3d.py and py2trt_rtfm.py are used to convert pytorch feature extractor & RTFM models to Torch TensorRT on Jetson Orin Nano

Citation

If you find this repo useful for your research, please consider citing our paper:

Pham, H.V., Tran, T.G., Le, C.D., Le, A.D., Vo, H.B. (2024). Benchmarking Jetson Edge Devices with an End-to-End Video-Based Anomaly Detection System. In: Arai, K. (eds) Advances in Information and Communication. FICC 2024. Lecture Notes in Networks and Systems, vol 920. Springer, Cham. https://doi.org/10.1007/978-3-031-53963-3_25

---