Skip to content

Conversation

@thepycoder
Copy link
Contributor

@thepycoder thepycoder commented Jan 12, 2023

This PR adds ClearML support to GenericLogger which is used by the instance segmentation and classification training scripts, also solves issue #10710

The feature list proposed in PR #8620 has been loosely followed, depending on what I had access to through the GenericLogger interface:

  • Current YOLO5 segmentation Tensorboard scalar metrics that can be visualized in a graph with ClearML -> so losses and segmentation mAPs (AP50:95, AP50, etc). The ones YOLO5 logs into "results.csv" file
  • opt.yaml, the config of the run
  • If possible, per class mAP metrics (we get those printed in the console when running inference, but not during training sadly)
  • Inference images on the test set (the one we get with YOLO automatically in the experiment folder)
  • Train batch images (the one we get with YOLO automatically in the experiment folder)
  • (this will be probably too much, but prediction masks on the test set in the COCO format)

The 2 features that are not yet included, aren't because I cannot access them via the existing logging interface. @glenn-jocher does it make sense to extend the GenericLogger for this?

🛠️ PR Summary

Made with ❤️ by Ultralytics Actions

🌟 Summary

Enhancements to ClearML logging support in Ultralytics YOLOv5.

📊 Key Changes

  • Added logging of label images and plots to ClearML during pretraining.
  • Improved how scalars (like loss, mAP) are logged to ClearML by introducing a new method log_scalars.
  • Enabled ClearML logging at the end of training to log a summary of results, all generated plots, and to differentiate between the best and last models.
  • ClearmlLogger initialization is now more robust, with exceptions caught for failed setup, and the project_name logic was updated to default to 'YOLOv5' when opt.project starts with 'runs/'.
  • Integrated matplotlib with ClearML to report plots directly.
  • Expanded the logging capabilities to include model weights, final summary metrics, and custom plots.
  • Code cleanup and removal of redundant or commented-out code blocks.

🎯 Purpose & Impact

  • Enhanced Monitoring: Users now have a more comprehensive and granular visualization of their models' training progress in ClearML.
  • Simplified Integration: Simplifying the codebase for ClearML integration makes maintenance easier and enhances user experience.
  • Better Experiment Tracking: Improved logging features allow for better tracking and comparison of different training runs, leading to more informed decision-making for model improvements.
  • Increased Compatibility: The upgrade ensures compatibility between YOLOv5 and the latest features offered by ClearML, facilitating a seamless workflow for those relying on this combination for their ML experiments.

@glenn-jocher
Copy link
Member

@thepycoder thanks for the PR! We'll take a look at this.

@AyushExel

@Robotatron
Copy link

@thepycoder I've tested it and everything works great, thank you!

ps . Could be off topic here but would you know if I have to configure clearML or edit your logger to also log the best model under "artefacts"? I am new to ClearML so could be a stupid question :)

@thepycoder thepycoder changed the title Feature/clearml segmentation classification Add segmentation and classification support for ClearML Jan 16, 2023
@pktiuk
Copy link

pktiuk commented Jun 22, 2023

Is there any chance for merging this?

@github-actions
Copy link
Contributor

github-actions bot commented Oct 3, 2023

👋 Hello there! We wanted to let you know that we've decided to close this pull request due to inactivity. We appreciate the effort you put into contributing to our project, but unfortunately, not all contributions are suitable or aligned with our product roadmap.

We hope you understand our decision, and please don't let it discourage you from contributing to open source projects in the future. We value all of our community members and their contributions, and we encourage you to keep exploring new projects and ways to get involved.

For additional resources and information, please see the links below:

Thank you for your contributions to YOLO 🚀 and Vision AI ⭐

@github-actions github-actions bot added the Stale Stale and schedule for closing soon label Oct 3, 2023
@pktiuk
Copy link

pktiuk commented Oct 3, 2023

Will this pr be closed because nobody had time to look at this?

@glenn-jocher
Copy link
Member

@pktiuk thanks for your contribution! We apologize for the delay in reviewing your pull request. Our team is working hard to review and address all incoming requests. We appreciate your patience in this matter. Rest assured, your pull request will be reviewed and evaluated soon. Thanks again for your contribution to YOLOv5!

@github-actions github-actions bot removed the Stale Stale and schedule for closing soon label Oct 4, 2023
@pktiuk
Copy link

pktiuk commented Nov 30, 2023

@glenn-jocher I hope so

@glenn-jocher
Copy link
Member

@pktiuk we greatly appreciate your understanding and patience. Thank you for your valuable contribution to the YOLOv5 repository! Your PR will be reviewed soon.

@glenn-jocher glenn-jocher merged commit 151c953 into ultralytics:master Jan 3, 2024
pleb631 pushed a commit to pleb631/yolov5-advanced that referenced this pull request Jan 6, 2024
…10752)

* Added ClearML instance segmentation and classification support

* Cleaned up ClearML plot output

* typos

* Log results as plots instead of debug samples

* [pre-commit.ci] auto fixes from pre-commit.com hooks

for more information, see https://pre-commit.ci

---------

Signed-off-by: Glenn Jocher <[email protected]>
Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>
Co-authored-by: Glenn Jocher <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants