Skip to content

Latest commit

 

History

History
133 lines (78 loc) · 5.37 KB

README.rst

File metadata and controls

133 lines (78 loc) · 5.37 KB
Raw

Multi-Stream Object Detection Pipeline

Overview

This GStreamer pipeline demonstrates object detection on multiple camera streams over RTSP protocol / Files.

All the streams are processed in parallel through the decode and scale phases, and enter the Hailo device frame by frame.

Afterwards post-process and drawing phases add the classified object and bounding boxes to each frame. The last step is to match each frame back to its respective stream and output all of them to the display.

Read more about RTSP: RTSP

Prerequisites

Preparations

In case of using RTSP cameras, configuration of the RTSP camera sources is required before running. open the multi_stream_detection_rtsp.sh in edit mode with your preferred editor. Configure the eight sources to match your own cameras.

readonly SRC_0="rtsp://<ip address>/?h264x=4 user-id=<username> user-pw=<password>"
readonly SRC_1="rtsp://<ip address>/?h264x=4 user-id=<username> user-pw=<password>"
etc..

Run the pipeline

./multi_stream_detection.sh

OR

./multi_stream_detection_rtsp.sh

  1. --show-fps Prints the fps to the output.

  2. --debug Uses gst-top to print time and memory consuming elements, saves the results as text and graph.

  3. --num-of-sources Sets the number of sources to use by given input. The default and recommended value in this pipeline is 12 (for files) or 8 (for camera streams over RTSP protocol).

  4. --set-live-source Use the actual source data given (example: /dev/video2). this flag is optional. if it's in use, num_of_sources is limited to 4.

  5. --tcp-address If specified, set the sink be a TCP (expected format is 'host:port').

  6. --network If specified, set the network to use. Choose from [yolov5, yolox, yolov8], default is yolov5.

  7. --device-count If specified, set the number of devices to use. Default (and maximum value) is the minimum between 4 and the number of devices on machine.

  8. --print-gst-launch Prints the ready gst-launch command without running it.

    NOTE: : When the debug flag is used and the app is running inside of a docker, exit the app by tying Ctrl+C in order to save the results. (Due to docker X11 display communication issues)

The output should look like:

readme_resources/example.jpg

Supported Networks

Overview of the Pipeline

These apps are based on our multi stream pipeline template

RTSP Specific Elements Used

  • rtspsrc Makes a connection to an rtsp server and read the data. Used as a src to get the video stream from rtsp-cameras.
  • rtph264depay Extracts h264 video from rtp packets.

Example of HailoRT Stream Multiplexer

  • This application shows the usage of the HailoRT Stream Multiplexer. This feature controls the time shared on the Hailo device between all streams. The Stream Multiplexer is enabled by the Hailonet scheduling-algorithm property when in use in multiple Hailonet elements that run the same HEF file. When the Stream Multiplexer is in use, there is no need to use funnel and streamiddemux like elements because the logic is handled internally.

Using Retraining to Replace Models

Note

It is recommended to first read the Retraining TAPPAS Models page.

Retraining Dockers (available on Hailo Model Zoo), can be used to replace the following models with ones that are trained on the users own dataset:

  • yolov8m

    • Retraining docker
      • For optimum compatibility and performance with TAPPAS, use for compilation the corresponding YAML file from above.
    • TAPPAS changes to replace model:
      • Update HEF_PATH on the .sh file

  • yolov5m

    • Retraining docker
      • For optimum compatibility and performance with TAPPAS, use for compilation the corresponding YAML file from above.
    • TAPPAS changes to replace model:
      • Update HEF_PATH on the .sh file

  • yolox_l_leaky

    • For optimum compatibility and performance with TAPPAS, use for compilation the corresponding YAML file from above.
    • TAPPAS changes to replace model:
      • Update HEF_PATH on the .sh file