ONNX export of YOLOv5 models with dynamic input size?

[sarmientoj24]

Search before asking

• I have searched the YOLOv5 issues and discussions and found no similar questions.

Question

It seems like that ONNX models can only take a fixed input size image. Is there a way for conversion that I can take dynamic input size for the ONNX model?

Additional

No response

[glenn-jocher]

@sarmientoj24 you might want to review export.py arguments:

yolov5/export.py

Line 557 in ea72b84

parser.add_argument('--dynamic', action='store_true', help='ONNX/TF: dynamic axes')

[sarmientoj24]

I tried that but I am still getting this when trying this input size 1468, 1968 as (W, H)

return self._sess.run(output_names, input_feed, run_options)
onnxruntime.capi.onnxruntime_pybind11_state.Fail: [ONNXRuntimeError] : 1 : FAIL : Non-zero status code returned while running Concat node. Name:'Concat_168' Status Message: concat.cc:156 PrepareForCompute Non concat axis dimensions must match: Axis 3 has mismatched dimensions of 1 and 2

[glenn-jocher]

@sarmientoj24 👋 hi, thanks for letting us know about this possible problem with YOLOv5 🚀. We've created a few short guidelines below to help users provide what we need in order to start investigating a possible problem.

How to create a Minimal, Reproducible Example

When asking a question, people will be better able to provide help if you provide code that they can easily understand and use to reproduce the problem. This is referred to by community members as creating a minimum reproducible example. Your code that reproduces the problem should be:

• ✅ Minimal – Use as little code as possible to produce the problem
• ✅ Complete – Provide all parts someone else needs to reproduce the problem
• ✅ Reproducible – Test the code you're about to provide to make sure it reproduces the problem

For Ultralytics to provide assistance your code should also be:

• ✅ Current – Verify that your code is up-to-date with GitHub master, and if necessary git pull or git clone a new copy to ensure your problem has not already been solved in master.
• ✅ Unmodified – Your problem must be reproducible using official YOLOv5 code without changes. Ultralytics does not provide support for custom code ⚠️.

If you believe your problem meets all the above criteria, please close this issue and raise a new one using the 🐛 Bug Report template with a minimum reproducible example to help us better understand and diagnose your problem.

Thank you! 😃

[sarmientoj24]

If I use something like (640, 768), it works.

[glenn-jocher]

@sarmientoj24 this works well for me. I'm unable to produce any errors with dynamic ONNX models at ~2000 pix

[glenn-jocher]

@sarmientoj24 please refer to my earlier message about supplying a reproducible example.

[sarmientoj24]

@glenn-jocher does your script resizes the image into an NxN image?

[sarmientoj24]

I have this as my script. I want it to be independent of the YOLOv5 script so I created.

My RGB image is (3, 1468, 1968) shape

import numpy as np
import onnxruntime
from PIL import Image

device = 'cuda'
model_path = 'model.onnx'
if "cuda" in device:
    providers = ["CUDAExecutionProvider"]
else:
    providers = ["CPUExecutionProvider"]

# Create Session
session = onnxruntime.InferenceSession(model_path, providers=providers)

img = Image.open(img_path)
img = np.array(img)

image_data = np.moveaxis(img, -1, 0)
image_data = image_data[np.newaxis, ...].astype(np.float32)
image_data /= 255

# Run prediction <--------- ERROR
output = session.run(
    [session.get_outputs()[0].name], {session.get_inputs()[0].name: image_data}
)[0]

[glenn-jocher]

@sarmientoj24 oh, I'm sorry, we don't debug custom code. Refer to #7268 (comment) to supply a reproducible example with unmodified YOLOv5 code.

[sarmientoj24]

@glenn-jocher I think the reason is that the shape should be divisible by 32. I already saw it from the code just now. It does resize by padding to the image. I'll just adapt some parts of the code like the letterbox

[sarmientoj24]

But if you pad to the image, the bbox coordinates must be realigned after getting the outputs, right?

[TengliEd]

@glenn-jocher If export to onnx by below command, there is an exception thrown: ONNX: export failure: Input, output and indices must be on the current device. It means that we can't dynamically export on gpu.
python export.py --include onnx --dynamic --device 0

[sarmientoj24]

@TengliEd I think it can still dynamically export on GPU. It's just there is a need to preprocess the image for it to be divisible by 32 as seen in detect.py.

[glenn-jocher]

@TengliEd not all combination of arguments are allowed. If you run into errors when adding argument simply don't use the arguments causing the errors, --device in your case.