Skip to content

Conversation

glenn-jocher
Copy link
Member

@glenn-jocher glenn-jocher commented Apr 24, 2022

Solution proposed in #7128 to TRT PyTorch Hub CUDA illegal memory errors.

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Improvement in device compatibility for AMP inferences within YOLOv5.

📊 Key Changes

  • Ensured that the dummy tensor p is on the same device as self.model when self.pt is False.

🎯 Purpose & Impact

  • Purpose: To fix a potential issue where the dummy tensor for device and type determination was not being consistently placed on the correct device (e.g., GPU or CPU).
  • Impact: Enhanced compatibility and stability when running Automatic Mixed Precision (AMP) inferences, preventing errors related to device mismatches. This change is particularly relevant when self.pt is False, ensuring that users who are performing inferences without a .pt model (PyTorch format) do not face device-related issues.

Solution proposed in #7128 to TRT PyTorch Hub CUDA illegal memory errors.
@glenn-jocher glenn-jocher self-assigned this Apr 24, 2022
@glenn-jocher
Copy link
Member Author

Fix works!

Screen Shot 2022-04-24 at 12 44 49 PM

@glenn-jocher glenn-jocher merged commit 950a85d into master Apr 24, 2022
@glenn-jocher glenn-jocher deleted the update/trt_hub branch April 24, 2022 19:45
@glenn-jocher glenn-jocher linked an issue Apr 24, 2022 that may be closed by this pull request
1 task
@glenn-jocher glenn-jocher linked an issue Apr 24, 2022 that may be closed by this pull request
1 task
@glenn-jocher glenn-jocher linked an issue Apr 24, 2022 that may be closed by this pull request
1 task
BjarneKuehl pushed a commit to fhkiel-mlaip/yolov5 that referenced this pull request Aug 26, 2022
Solution proposed in ultralytics#7128 to TRT PyTorch Hub CUDA illegal memory errors.
ctjanuhowski pushed a commit to ctjanuhowski/yolov5 that referenced this pull request Sep 8, 2022
Solution proposed in ultralytics#7128 to TRT PyTorch Hub CUDA illegal memory errors.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

AutoShape Usage Cuda illegal memory access when running inference on *.engine Batch Inference to get bounding box coordinates using TensorRT
1 participant