How to check the "mAP" value from the detect.py?

[jaskiratsingh2000]

Hi @glenn-jocher I just want to ask that I want to check the mAP value of the TFlite model from the detect.py Is there any way I can get that value by passing some kind of parser?

FYI, I don't want to use the val.py script since it doesn't have tflite model implementation for mAP. Can you please let me know?

[glenn-jocher]

@jaskiratsingh2000 val.py is used for metrics computation on a dataset yaml:

python val.py --data coco128.yaml --weights yolov5s.pt

[jaskiratsingh2000]

@glenn-jocher I think you haven't understood my question I actually want to measure the mAP value of tf.py model hence val.py doesn't provide the code implementation and I don't know how to do that is why I said that since mAP is a subset of detect.py so how can we do that

[glenn-jocher]

@jaskiratsingh2000 yes I understand the feature request. This would be a useful feature to have. I will add this to our TODO queue to task future dev work on this, but it won't be ready anytime soon since we have many existing tasks to complete first.

The best solution is probably an extensible multi-backend inference engine that val.py and detect.py can both call, which we can keep adding backends to, i.e. cv2 DNN, CoreML, etc.

[jaskiratsingh2000]

@glenn-jocher How long I could expect this feature to come up?

In the meantime can you just try explaining me in bit more detail that what exactly I need to do to find mAP using detect.py ?

[glenn-jocher]

@jaskiratsingh2000 there's no timeline for this. I'm sorry I can't help with code customizations.

[jaskiratsingh2000]

@glenn-jocher Okay no worries. Also can you please look into my another issue as well which talks about format of pytorch model?

…

On Thu, 7 Oct 2021, 1:39 am Glenn Jocher, ***@***.***> wrote: @jaskiratsingh2000 <https://github.com/jaskiratsingh2000> there's no timeline for this. I'm sorry I can't help with code customizations. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#5069 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACOBHK6MHFINLDKJK4V3373UFSUI7ANCNFSM5FPCKQPA> .

[github-actions]

👋 Hello, this issue has been automatically marked as stale because it has not had recent activity. Please note it will be closed if no further activity occurs.

Access additional YOLOv5 🚀 resources:

• Wiki – https://github.com/ultralytics/yolov5/wiki
• Tutorials – https://docs.ultralytics.com/yolov5
• Docs – https://docs.ultralytics.com

Access additional Ultralytics ⚡ resources:

• Ultralytics HUB – https://ultralytics.com/hub
• Vision API – https://ultralytics.com/yolov5
• About Us – https://ultralytics.com/about
• Join Our Team – https://ultralytics.com/work
• Contact Us – https://ultralytics.com/contact

Feel free to inform us of any other issues you discover or feature requests that come to mind in the future. Pull Requests (PRs) are also always welcomed!

Thank you for your contributions to YOLOv5 🚀 and Vision AI ⭐!

[glenn-jocher]

@jaskiratsingh2000 good news 😃! Your original issue may now be fixed ✅ in PR #5549. This PR consolidates all backends into a single YOLOv5 DetectMultiBackend() class. Supports following model formats:

• PyTorch *.pt
• TorchScript *.torchscript.pt
• CoreML *.mlmodel
• TensorFlow saved_model
• TensorFlow *.pb
• TensorFlow Lite *.tflite
• TensorFlow.js web_model
• ONNX Runtime *.onnx
• OpenCV DNN *.onnx

Usage

# Export
python export.py --weights yolov5s.pt --include tflite

# Inference
python detect.py --weights yolov5s.tflite
python val.py --weights yolov5s.tflite

To receive this update:

• Git – git pull from within your yolov5/ directory or git clone https://github.com/ultralytics/yolov5 again
• PyTorch Hub – Force-reload model = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True)
• Notebooks – View updated notebooks
• Docker – sudo docker pull ultralytics/yolov5:latest to update your image

Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!