Why the inference speed when the image size is 224 is slower than the image size is 640？

[daydayup-bit]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

When i enter commandpython val.py --data container_yolo/NEU-DET/data.yaml --weights runs/train/v5s_all_new/weights/best.pt --imgsz 640，time of inference is 2.4ms，the result：

when i enter commandpython val.py --data container_yolo/NEU-DET/data.yaml --weights runs/train/v5s_all_new/weights/best.pt --imgsz 224，time of inference is 4.1ms,which is slower than inputing 640. the result:

Why the inference speed when the image size is 224 is slower than the image size is 640？
And, what are the main effects of these three parameters(pre-process,inference,NMS).Especially what affects the inference time？

Additional

No response

[github-actions]

👋 Hello @daydayup-bit, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email Glenn Jocher at glenn.jocher@ultralytics.com.

Requirements

Python>=3.6.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

$ git clone https://github.com/ultralytics/yolov5
$ cd yolov5
$ pip install -r requirements.txt

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Google Colab and Kaggle notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training (train.py), validation (val.py), inference (detect.py) and export (export.py) on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@daydayup-bit 👋 Hello! Thanks for asking about inference speed issues. YOLOv5 🚀 can be run on CPU (i.e. --device cpu, slow) or GPU if available (i.e. --device 0, faster). You can determine your inference device by viewing the YOLOv5 line printed to screen:

If you would like to increase your inference speed some options are:

• Increase --batch-size (YOLOv5 PyTorch Hub and val.py only)
• Reduce --img-size, i.e. 1280 -> 640 -> 320
• Reduce model size, i.e. YOLOv5x -> YOLOv5l -> YOLOv5m -> YOLOv5s -> YOLOv5n
• Use a faster GPUs, i.e.: P100 -> V100 -> A100
• Use a free GPU backends with up to 16GB of CUDA memory:

detect.py inference

python detect.py --weights yolov5s.pt --img 640 --conf 0.25 --source data/images/

YOLOv5 PyTorch Hub inference

import torch

# Model
model = torch.hub.load('ultralytics/yolov5', 'yolov5s')

# Images
dir = 'https://ultralytics.com/images/'
imgs = [dir + f for f in ('zidane.jpg', 'bus.jpg')]  # batch of images

# Inference
results = model(imgs)
results.print()  # or .show(), .save()
# Speed: 631.5ms pre-process, 19.2ms inference, 1.6ms NMS per image at shape (2, 3, 640, 640)

[daydayup-bit]

@glenn-jocher Thank you very much for your answer. You said that reducing --img-size can increase the speed of inference, but on my machine, I found that this is not the case. When I use yolov5s to run detect.py on the GPU and enter a different --img-size, I get the following result.

--img-size	inference time/per img
1280	10.6ms
640	9.1ms
480	8.4ms
320	9.9ms
224	9.5ms

As you can see, the inference time ranking is not the 224<320<480<640<1280 we expected, but 480<640<224<320<1280. I used a total of 180 images for detect. Why does this happen? I am very confused.

[glenn-jocher]

@daydayup-bit I can't speak for your specific environment, but if your times don't improve then there's a constraint somewhere in your workflow unrelated to GPU or image size.

[daydayup-bit]

Okay, thank you for your reply.

[kuonumber]

@glenn-jocher Do you mean if I train model in larger batch size, I will get less inference time?

[glenn-jocher]

@kuonumber @daydayup-bit we recently published a full inference speed vs batch size study here:
https://community.ultralytics.com/t/yolov5-study-batch-size-vs-speed

[kuonumber]

@glenn-jocher WOW THAT'S REALLY COOL!