Apple MPS inference with detect.py

[lightonthefloor]

Search before asking

• I have searched the YOLOv5 issues and found no similar bug report.

YOLOv5 Component

Detection

Bug

When I run the command: python detect.py --source 0 --device mps, there is no detections. But when I run by using cpu, it has.

Environment

• Yolo v5-6.2
• macOS 12.6
• conda 4.14
• python 3.10.4

Minimal Reproducible Example

No response

Additional

No response

Are you willing to submit a PR?

• Yes I'd like to help by submitting a PR!

[github-actions]

👋 Hello @lightonthefloor, thank you for your interest in YOLOv5 🚀! Please visit our ⭐️ Tutorials to get started, where you can find quickstart guides for simple tasks like Custom Data Training all the way to advanced concepts like Hyperparameter Evolution.

If this is a 🐛 Bug Report, please provide screenshots and minimum viable code to reproduce your issue, otherwise we can not help you.

If this is a custom training ❓ Question, please provide as much information as possible, including dataset images, training logs, screenshots, and a public link to online W&B logging if available.

For business inquiries or professional support requests please visit https://ultralytics.com or email support@ultralytics.com.

Requirements

Python>=3.7.0 with all requirements.txt installed including PyTorch>=1.7. To get started:

git clone https://github.com/ultralytics/yolov5  # clone
cd yolov5
pip install -r requirements.txt  # install

Environments

YOLOv5 may be run in any of the following up-to-date verified environments (with all dependencies including CUDA/CUDNN, Python and PyTorch preinstalled):

• Notebooks with free GPU:
• Google Cloud Deep Learning VM. See GCP Quickstart Guide
• Amazon Deep Learning AMI. See AWS Quickstart Guide
• Docker Image. See Docker Quickstart Guide

Status

If this badge is green, all YOLOv5 GitHub Actions Continuous Integration (CI) tests are currently passing. CI tests verify correct operation of YOLOv5 training, validation, inference, export and benchmarks on MacOS, Windows, and Ubuntu every 24 hours and on every commit.

[glenn-jocher]

@lightonthefloor MPS inference is waiting on a few remaining operations to be integrated into torch nightly, so you should receive an error similar to this:

python detect.py --source 0 --device mps

detect: weights=yolov5s.pt, source=0, data=data/coco128.yaml, imgsz=[640, 640], conf_thres=0.25, iou_thres=0.45, max_det=1000, device=mps, view_img=False, save_txt=False, save_conf=False, save_crop=False, nosave=False, classes=None, agnostic_nms=False, augment=False, visualize=False, update=False, project=runs/detect, name=exp, exist_ok=False, line_thickness=3, hide_labels=False, hide_conf=False, half=False, dnn=False, vid_stride=1
YOLOv5 🚀 v6.2-154-g5c34c4c Python-3.9.0 torch-1.12.1 MPS

Fusing layers... 
YOLOv5s summary: 213 layers, 7225885 parameters, 0 gradients
1/1: 0...  Success (inf frames 1920x1080 at 15.00 FPS)

Traceback (most recent call last):
  File "/Users/glennjocher/PycharmProjects/yolov5/detect.py", line 255, in <module>
    main(opt)
  File "/Users/glennjocher/PycharmProjects/yolov5/detect.py", line 250, in main
    run(**vars(opt))
  File "/Users/glennjocher/PycharmProjects/yolov5/venv39/lib/python3.9/site-packages/torch/autograd/grad_mode.py", line 27, in decorate_context
    return func(*args, **kwargs)
  File "/Users/glennjocher/PycharmProjects/yolov5/detect.py", line 126, in run
    pred = non_max_suppression(pred, conf_thres, iou_thres, classes, agnostic_nms, max_det=max_det)
  File "/Users/glennjocher/PycharmProjects/yolov5/utils/general.py", line 843, in non_max_suppression
    x = x[xc[xi]]  # confidence
NotImplementedError: The operator 'aten::index.Tensor' is not current implemented for the MPS device. If you want this op to be added in priority during the prototype phase of this feature, please comment on https://github.com/pytorch/pytorch/issues/77764. As a temporary fix, you can set the environment variable `PYTORCH_ENABLE_MPS_FALLBACK=1` to use the CPU as a fallback for this op. WARNING: this will be slower than running natively on MPS.

[lightonthefloor]

Nope. I have already set this environment variable.

[lightonthefloor]

`(yolo-v5_test) trisoil@Trisoils-MacBook-Pro yolov5-master % python detect.py --source 0 --device mps
detect: weights=yolov5s.pt, source=0, data=data/coco128.yaml, imgsz=[640, 640], conf_thres=0.25, iou_thres=0.45, max_det=1000, device=mps, view_img=False, save_txt=False, save_conf=False, save_crop=False, nosave=False, classes=None, agnostic_nms=False, augment=False, visualize=False, update=False, project=runs/detect, name=exp, exist_ok=False, line_thickness=3, hide_labels=False, hide_conf=False, half=False, dnn=False
YOLOv5 🚀 2022-9-17 Python-3.10.4 torch-1.12.1 MPS

Fusing layers...
YOLOv5s summary: 213 layers, 7225885 parameters, 0 gradients
[ WARN:0@1.789] global /private/var/folders/nz/j6p8yfhx1mv_0grj5xl4650h0000gp/T/abs_c6qq9eqk9d/croots/recipe/opencv-suite_1663872527491/work/modules/videoio/src/cap_gstreamer.cpp (862) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
1/1: 0... Success (inf frames 1920x1080 at 15.00 FPS)

/Users/trisoil/OneDrive - bupt.edu.cn/College Learning/Robocon/yolov5-master/utils/general.py:838: UserWarning: The operator 'aten::index.Tensor' is not currently supported on the MPS backend and will fall back to run on the CPU. This may have performance implications. (Triggered internally at /Users/runner/work/_temp/anaconda/conda-bld/pytorch_1659484612588/work/aten/src/ATen/mps/MPSFallback.mm:11.)
x = x[xc[xi]] # confidence

(:84503): GStreamer-CRITICAL **: 19:13:30.121: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed
[ WARN:0@3.850] global /private/var/folders/nz/j6p8yfhx1mv_0grj5xl4650h0000gp/T/abs_c6qq9eqk9d/croots/recipe/opencv-suite_1663872527491/work/modules/videoio/src/cap_gstreamer.cpp (2180) open OpenCV | GStreamer warning: cannot link elements
0: 384x640 (no detections), 167.8ms
0: 384x640 (no detections), 17.8ms
0: 384x640 (no detections), 14.6ms
0: 384x640 (no detections), 15.4ms
`

[glenn-jocher]

@lightonthefloor I'm getting a different error message if I use the fallback. Do you see the same thing? I'm using latest nightly.

PYTORCH_ENABLE_MPS_FALLBACK=1 python detect.py --device mps

[lightonthefloor]

No. And I am not using the latest nightly. My torch version is 1.12.1

[glenn-jocher]

@lightonthefloor if you really want to use MPS today you can add this fix (insert predictions = predictions.cpu() in NMS where shown). This eliminates the need to pass the fallback flag, but it passes all predictions to CPU before NMS, which also takes time. My metrics on an M2 MacBook with this change are here. Note also that MPS is slow for the first image and then very fast on subsequent images of the same size.

detect: weights=yolov5s.pt, source=data/images, data=data/coco128.yaml, imgsz=[640, 640], conf_thres=0.25, iou_thres=0.45, max_det=1000, device=mps, view_img=False, save_txt=False, save_conf=False, save_crop=False, nosave=False, classes=None, agnostic_nms=False, augment=False, visualize=False, update=False, project=runs/detect, name=exp, exist_ok=False, line_thickness=3, hide_labels=False, hide_conf=False, half=False, dnn=False, vid_stride=1
YOLOv5 🚀 v6.2-171-gbd9c0c4 Python-3.10.6 torch-1.13.0.dev20220926 MPS

Fusing layers... 
YOLOv5s summary: 213 layers, 7225885 parameters, 0 gradients
image 1/3 /Users/glennjocher/PycharmProjects/yolov5/data/images/bus.jpg: 640x480 9 persons, 2 buss, 143.4ms
image 2/3 /Users/glennjocher/PycharmProjects/yolov5/data/images/zidane.jpg: 384x640 5 persons, 5 ties, 189.1ms
video 3/3 (1/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 (no detections), 184.8ms
video 3/3 (2/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 (no detections), 12.8ms
video 3/3 (3/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 (no detections), 11.2ms
video 3/3 (4/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 (no detections), 13.9ms
video 3/3 (5/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 (no detections), 16.7ms
video 3/3 (6/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 (no detections), 8.4ms
video 3/3 (7/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 (no detections), 8.5ms
video 3/3 (8/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 (no detections), 7.8ms
video 3/3 (9/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 (no detections), 7.6ms
video 3/3 (10/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 1 person, 8.1ms
video 3/3 (11/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 (no detections), 7.6ms
video 3/3 (12/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 (no detections), 8.1ms
video 3/3 (13/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 3 persons, 7.7ms
video 3/3 (14/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 3 persons, 8.0ms
video 3/3 (15/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 3 persons, 11.3ms
video 3/3 (16/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 3 persons, 7.8ms
video 3/3 (17/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 (no detections), 7.5ms
video 3/3 (18/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 1 person, 8.1ms
video 3/3 (19/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 2 trucks, 8.0ms
video 3/3 (20/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 11 persons, 8.0ms
video 3/3 (21/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 9 persons, 7.8ms
video 3/3 (22/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 9 persons, 7.9ms
video 3/3 (23/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 10 persons, 7.7ms
video 3/3 (24/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 (no detections), 7.9ms
video 3/3 (25/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 (no detections), 7.8ms
video 3/3 (26/26) /Users/glennjocher/PycharmProjects/yolov5/data/images/IMG_2590_rotate180.MOV: 640x384 3 persons, 8.0ms
Speed: 1.2ms pre-process, 26.5ms inference, 9.7ms NMS per image at shape (1, 3, 640, 640)
Results saved to runs/detect/exp27