INCREASING NMS SPEED

[glenn-jocher]

Non Maximal Suppression (NMS) of bounding boxes is a significant speed constraint during testing. I am opening this issue to try to determine options for speeding up this operation. I am going to compare the default NMS method 'MERGE' with two newly available PyTorch methods. If anyone has any additional methods we could test, please post here.

yolov3/utils/utils.py

Line 456 in cadd2f7

def non_max_suppression(prediction, conf_thres=0.5, nms_thres=0.5):

The test code is below. Hardware is a 2080Ti.

python3 test.py --weights ultralytics68.pt --nms-thres 0.6 --img-size 512 --device 0

UPDATE: THESE ARE OLD RESULTS, SEE BOTTOM OF THREAD FOR IMPROVED RESULTS

	Speed mm:ss	COCO mAP @0.5...0.95	COCO mAP @0.5
ultralytics 'OR'	8:20	39.7	60.3
ultralytics 'AND'	7:38	39.6	60.1
ultralytics 'SOFT'	12:00	39.1	58.7
ultralytics 'MERGE'	11:25	40.2	60.4
torchvision.ops.boxes.nms()	5:08	39.7	60.3
torchvision.ops.boxes.batched_nms()	6:00	39.7	60.3

[glenn-jocher]

Results of the test is that torchvision.ops.boxes.nms() is fastest but not the highest mAP. Ultralytics MERGE method increases AP + 0.5, so I will leave it for testing (when calling test.py directly using --conf-thres 0.001), and use torchvision.ops.boxes.nms() for calculating mAP when training using --conf-thres 0.10 (to increase training speed).

yolov3/utils/utils.py

Lines 513 to 517 in 1e9ddc5

	# Set NMS method https://github.com/ultralytics/yolov3/issues/679
	# 'OR', 'AND', 'MERGE', 'VISION', 'VISION_BATCHED'
	method = 'MERGE' if conf_thres <= 0.01 else 'VISION' # MERGE is highest mAP, VISION is fastest

[FranciscoReveriano]

I will look more into this during the weekend.

[developer0hye]

great works!

[omizonly]

torchvision. ops implements operators that are specific for Computer Vision. Those operators currently do not support TorchScript. Performs non-maximum suppression (NMS) on the boxes according to their intersection-over-union (IoU)

AttributeError: module 'torchvision' has no attribute 'ops'

what should I do?

[glenn-jocher]

@omizonly what is your use case for TorchScript?

[omizonly]

@omizonly what is your use case for TorchScript?

tensorflow= 1.3.1

[glenn-jocher]

@omizonly I don't understand, can you elaborate? This repo only runs PyTorch, and exports to ONNX for onward use in other formats, however we clearly can not support you with problems in those other formats. I suggest you raise an issue on the PyTorch or TF repos.

[glenn-jocher]

I'll close this issue for now as the original issue appears to have been resolved, and/or no activity has been seen for some time. Feel free to comment if this is not the case.

[glenn-jocher]

Quick update with latest code on one T4 GPU. Second line is current default.

python3 test.py --weights yolov3-spp-ultralytics.pt --cfg yolov3-spp.cfg --img 608

	Time sec/image	Time mm:ss	COCO mAP @0.5...0.95	COCO mAP @0.5
'vision_batched', multi_cls=False	43ms	3:36	40.2	60.4
'vision_batched', multi_cls=True	48ms	4:01	40.9	61.4
'merge', multi_cls=True	172ms	14:23	41.3	61.7

[FranciscoReveriano]

Is there a way to make the model print the JSON file if it detects an object regardless of classification?