From 813bfc304ad4935b1157d19363ff8c30754a61be Mon Sep 17 00:00:00 2001 From: zhiqiang Date: Mon, 4 Oct 2021 00:54:56 +0800 Subject: [PATCH 1/2] Add notes about the exported ONNX model --- deployment/onnxruntime/README.md | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/deployment/onnxruntime/README.md b/deployment/onnxruntime/README.md index b311dcca..fde1e1fa 100644 --- a/deployment/onnxruntime/README.md +++ b/deployment/onnxruntime/README.md @@ -11,6 +11,14 @@ The ONNXRuntime inference for `yolort`, both GPU and CPU are supported. *We didn't impose too strong restrictions on the versions of dependencies.* +## Features + +The default `ONNX` model exported with `yolort` differs from the official one in following three main ways. + +- The exported `ONNX` graph now supports dynamic shapes, and we use `(3, H, W)` (for example `(3, 640, 640)`) as the input shape. +- We embed the pre-processing ([`letterbox`](https://github.com/ultralytics/yolov5/blob/9ef94940aa5e9618e7e804f0758f9a6cebfc63a9/utils/augmentations.py#L88-L118)) into the graph as well. We only require the input image to be in the `RGB` channel, and to be rescaled to `float32 [0-1]` from general `uint [0-255]`. The main logic we use to implement this mechanism is below. (And [this](https://github.com/zhiqwang/yolov5-rt-stack/blob/b9c67205a61fa0e9d7e6696372c133ea0d36d9db/yolort/models/transform.py#L210-L234) plays the same role of the official letterbox, but there will be a little difference in accuracy now.) +- We embed the post-processing (`nms`) into the model graph, which performs the same task as [`non_max_suppression`](https://github.com/ultralytics/yolov5/blob/fad57c29cd27c0fcbc0038b7b7312b9b6ef922a8/utils/general.py#L532-L623) except for the format of the inputs. (And here the `ONNX` graph is required to be dynamic.) + ## Usage 1. First, Setup the environment variable. @@ -35,7 +43,7 @@ The ONNXRuntime inference for `yolort`, both GPU and CPU are supported. [--simplify] ``` - Afterwards, you can see that a new pair of ONNX models ("best.onnx" and "best.sim.onnx") has been generated in the directory of "best.pt". + And then, you can find that a new pair of ONNX models ("best.onnx" and "best.sim.onnx") has been generated in the directory of "best.pt". 1. \[Optional\] Quick test with the ONNXRuntime Python interface. From 99f2071a8f5466846dfdfbf76a504abcad2cdb70 Mon Sep 17 00:00:00 2001 From: zhiqiang Date: Mon, 4 Oct 2021 00:57:30 +0800 Subject: [PATCH 2/2] Minor fixes --- deployment/onnxruntime/README.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/deployment/onnxruntime/README.md b/deployment/onnxruntime/README.md index fde1e1fa..7bbf9a68 100644 --- a/deployment/onnxruntime/README.md +++ b/deployment/onnxruntime/README.md @@ -13,10 +13,10 @@ The ONNXRuntime inference for `yolort`, both GPU and CPU are supported. ## Features -The default `ONNX` model exported with `yolort` differs from the official one in following three main ways. +The `ONNX` model exported with `yolort` differs from the official one in the following three ways. -- The exported `ONNX` graph now supports dynamic shapes, and we use `(3, H, W)` (for example `(3, 640, 640)`) as the input shape. -- We embed the pre-processing ([`letterbox`](https://github.com/ultralytics/yolov5/blob/9ef94940aa5e9618e7e804f0758f9a6cebfc63a9/utils/augmentations.py#L88-L118)) into the graph as well. We only require the input image to be in the `RGB` channel, and to be rescaled to `float32 [0-1]` from general `uint [0-255]`. The main logic we use to implement this mechanism is below. (And [this](https://github.com/zhiqwang/yolov5-rt-stack/blob/b9c67205a61fa0e9d7e6696372c133ea0d36d9db/yolort/models/transform.py#L210-L234) plays the same role of the official letterbox, but there will be a little difference in accuracy now.) +- The exported `ONNX` graph now supports dynamic shapes, and we use `(3, H, W)` as the input shape (for example `(3, 640, 640)`). +- We embed the pre-processing ([`letterbox`](https://github.com/ultralytics/yolov5/blob/9ef94940aa5e9618e7e804f0758f9a6cebfc63a9/utils/augmentations.py#L88-L118)) into the graph as well. We only require the input image to be in the `RGB` channel, and to be rescaled to `float32 [0-1]` from general `uint [0-255]`. The main logic we use to implement this mechanism is below. (And [this](https://github.com/zhiqwang/yolov5-rt-stack/blob/b9c67205a61fa0e9d7e6696372c133ea0d36d9db/yolort/models/transform.py#L210-L234) plays the same role of the official `letterbox`, but there will be a little difference in accuracy now.) - We embed the post-processing (`nms`) into the model graph, which performs the same task as [`non_max_suppression`](https://github.com/ultralytics/yolov5/blob/fad57c29cd27c0fcbc0038b7b7312b9b6ef922a8/utils/general.py#L532-L623) except for the format of the inputs. (And here the `ONNX` graph is required to be dynamic.) ## Usage