-
-
Notifications
You must be signed in to change notification settings - Fork 16.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added support for fp16 (half) to export.py #3010
Conversation
@hodovo thanks for the PR! I've minimized the changes a bit but otherwise looks good. This work conflicts with an earlier PR in progress #2982, so we will merge the earlier PR first hopefully early this week, and then once #2982 is merged we will merge master in this one, resolving conflicts manually. |
Sounds good! Your changes look much nicer :) Let me know if you need anything. Thank you. |
@hodovo PR is merged. Thank you for your contributions! |
My pleasure. Thank you for the awesome platform you have built :) |
Does torchscript work properly in half precision? In this issue it is stated that half precison of convolution is not implemented. A simple testing code for the exported models may be nice for this case. |
@xonobo Yes it does. That issue is specifically for CPU and not GPU. The implementation in this pull request checks to ensure that the export device is a CUDA device. However, that doesn't enforce how the user utilizes the model afterwards. |
* Added support for fp16 (half) to export.py * minimize code additions Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Added support for fp16 (half) to export.py * minimize code additions Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
* Added support for fp16 (half) to export.py * minimize code additions Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com> (cherry picked from commit e2a80c6)
* Added support for fp16 (half) to export.py * minimize code additions Co-authored-by: Glenn Jocher <glenn.jocher@ultralytics.com>
This adds support for the --half argument when exporting a model.
Considerations
I have only tested this with torchscript. It might be a good idea to only support this for torchscript if you know of any problems with ONNX/CoreML and fp16. My focus is on torchscript. I can easily modify this pull request to load the model separately as half for only torchscript and also create a separate img for the trace.
Let me know your thoughts :)
🛠️ PR Summary
Made with ❤️ by Ultralytics Actions
🌟 Summary
Added FP16 half-precision export option to YOLOv5 model exporter. 🚀
📊 Key Changes
--half
argument to the model export script enabling FP16 half-precision exports. 🎛️--half
flag is used. 🔧🎯 Purpose & Impact
--half
flag allows users to export their models with reduced precision, which can lead to performance gains on compatible hardware (mostly GPUs) by reducing the computational workload. 📈