Add support for exporting models with half precision.

[hodovo]

🚀 Feature

Models are currently exported in FP32. Adding support for FP16 (half) export will allow for greater flexibility in model performance.

Motivation

To improve the performance of torchscript models.

Pitch

Ideally, this should be a very simple feature if I am not missing anything. I have integrated it on my local for exporting and I can make a pull request. Essentially, I added a "half" argument as follows:

parser.add_argument('--half', action='store_true', help='export with half precision')

When the model is loaded, we do as follows:

if opt.half:
        model = model.half()

Then, when we create the empty image to run through the model, we do as follows:

if opt.half:
        img = img.half()

I have only tested this for torchscript and confirmed it is working. It improves the inference throughput by around 33% on a RTX 3090 with a batch size of 16 and image size of 320.

Please let me know what you think. Thank you for taking the time to read my issue :)

[github-actions]

👋 Hello @hodovo, thank you for your interest in 🚀 YOLOv5! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://www.ultralytics.com or email Glenn Jocher at glenn.jocher@ultralytics.com.

Requirements

Python 3.8 or later with all requirements.txt dependencies installed, including torch>=1.7. To install run:

$ pip install -r requirements.txt

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), testing (test.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@hodovo yes this is a good idea! Please submit a PR, thank you!

[hodovo]

I just made the pull request #3010 :)

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.