This project provides the code and results for 'Learning Class Prototypes for Unified Sparse-Supervised 3D Object Detection', CVPR 2025 Highlight.

Anchors: Yun Zhu, Le Hui, Hang Yang, Jianjun Qian, Jin Xie*, Jian Yang

PaperLink: https://arxiv.org/abs/2503.21099

Both indoor and outdoor scene perceptions are essential for embodied intelligence. However, current sparse supervised 3D object detection methods focus solely on outdoor scenes without considering indoor settings. To this end, we propose a unified sparse supervised 3D object detection method for both indoor and outdoor scenes through learning class prototypes to effectively utilize unlabeled objects. Specifically, we first propose a prototype-based object mining module that converts the unlabeled object mining into a matching problem between class prototypes and unlabeled features. By using optimal transport matching results, we assign prototype labels to high-confidence features, thereby achieving the mining of unlabeled objects. We then present a multi-label cooperative refinement module to effectively recover missed detections through pseudo label quality control and prototype label cooperation. Experiments show that our method achieves state-of-the-art performance under the one object per scene sparse supervised setting across indoor and outdoor datasets. With only one labeled object per scene, our method achieves about 78%, 90%, and 96% performance compared to the fully supervised detector on ScanNet V2, SUN RGB-D, and KITTI, respectively, highlighting the scalability of our method.

• To install the environment, we follow SPGroup3D.

• All the CPDet3D-related code locates in the folder projects_sparse.

• Follow the mmdetection3d data preparation protocol described in scannet, sunrgbd.

• We provide the indoor sparse supervised split of scannet_infos_train.pkl and sunrgbd_infos_train.pkl on GoogleDrive.
• Otherwise, you can generate your own pkl by create_sparse_infos.

To start training, run train with CPDet3D configs, which includes two stages.

Test pre-trained model using test with CPDet3D configs.

Due to the size of these datasets and the randomness that inevitably exists in the model, the results on these datasets fluctuate significantly. It's normal for results to fluctuate within a range.

If you find this work useful for your research, please cite our paper:

@inproceedings{cpdet3d,
  title={Learning Class Prototypes for Unified Sparse-Supervised 3D Object Detection},
  author={Yun Zhu, Le Hui, Hang Yang, Jianjun Qian, Jin Xie, Jian Yang},
  booktitle={CVPR},
  year={2025}
}

This project is based on the following codebases.

• mmdetection3D
• TR3D
• ProtoSeg
• DQS3D

If you find this project helpful, please also cite the codebases above. Thanks.