Single class inference, class score is still 0.98 but not 1

[yuanyuangoo]

Before submitting a bug report, please be aware that your issue must be reproducible with all of the following,
otherwise it is non-actionable, and we can not help you:

• Current repo: run git fetch && git status -uno to check and git pull to update repo
• Common dataset: coco.yaml or coco128.yaml
• Common environment: Colab, Google Cloud, or Docker image. See https://github.com/ultralytics/yolov5#environments

If this is a custom dataset/training question you must include your train*.jpg, val*.jpg and results.png
figures, or we can not help you. You can generate these with utils.plot_results().

🐛 Bug

A clear and concise description of what the bug is.

To Reproduce (REQUIRED)

Input:

Just train a model using,

python train.py --single-cls

Output:

a model in runs/train/exp/weights/best.pt

Expected behavior

run debug mode

python detect.py --weights runs/train/exp/weights/best.pt

we should see that in utils/general.py line 655, print x[:,5] which is the cls_conf and it should be all 1 in single class training, but now they are tensor([0.99923, 0.99898, 0.99948, ..., 0.97183, 0.97135, 0.97071], device='cuda:0')

Environment

If applicable, add screenshots to help explain your problem.

• OS: Linux Mint
• GPU: 2080 Ti

Additional context

this problem would affect the overall confidence of detection

n.

[glenn-jocher]

@yuanyuangoo yes I see. I'm not sure that it's going to affect model performance much, but if you'd like you can experiment with updating output biases here to better handle single-class configurations. If you come up with a solution that works well, and does not affect multi-class training, then please let us know or submit a PR to help future users. Thank you!

yolov5/models/yolo.py

Lines 196 to 205 in a4fece8

	def _initialize_biases(self, cf=None): # initialize biases into Detect(), cf is class frequency
	# https://arxiv.org/abs/1708.02002 section 3.3
	# cf = torch.bincount(torch.tensor(np.concatenate(dataset.labels, 0)[:, 0]).long(), minlength=nc) + 1.
	m = self.model[-1] # Detect() module
	for mi, s in zip(m.m, m.stride): # from
	b = mi.bias.view(m.na, -1) # conv.bias(255) to (3,85)
	b.data[:, 4] += math.log(8 / (640 / s) ** 2) # obj (8 objects per 640 image)
	b.data[:, 5:] += math.log(0.6 / (m.nc - 0.99)) if cf is None else torch.log(cf / cf.sum()) # cls
	mi.bias = torch.nn.Parameter(b.view(-1), requires_grad=True)

[glenn-jocher]

@yuanyuangoo current output is set to

torch.sigmoid(torch.tensor([math.log(0.6 / (1 - 0.99))])) = tensor([0.9836])

for nc=1

[glenn-jocher]

@yuanyuangoo some quick experiments show we could add another few decimal places to this value without impacting multi-class results much, i.e.:

torch.sigmoid(torch.tensor([math.log(0.6 / (1 - 0.9999))]))
Out[16]: tensor([0.9998])

torch.sigmoid(torch.tensor([math.log(0.6 / (5 - 0.99))]))
Out[17]: tensor([0.1302])
torch.sigmoid(torch.tensor([math.log(0.6 / (5 - 0.9999))]))
Out[18]: tensor([0.1304])

Can you want to submit a PR with this change?

torch.sigmoid(torch.tensor([math.log(0.6 / (1 - 0.99999))]))
Out[19]: tensor([1.0000])

[glenn-jocher]

TODO: single-class bias init update

[glenn-jocher]

@yuanyuangoo good news 😃! Your original issue may now be fixed ✅ in PR #5520.

Since you did not submit a PR I did this myself. Numerical precision is now increased and torch returns 1.0000 probability for single-class datasets with this PR.

To receive this update:

• Git – git pull from within your yolov5/ directory or git clone https://github.com/ultralytics/yolov5 again
• PyTorch Hub – Force-reload model = torch.hub.load('ultralytics/yolov5', 'yolov5s', force_reload=True)
• Notebooks – View updated notebooks
• Docker – sudo docker pull ultralytics/yolov5:latest to update your image

Thank you for spotting this issue and informing us of the problem. Please let us know if this update resolves the issue for you, and feel free to inform us of any other issues you discover or feature requests that come to mind. Happy trainings with YOLOv5 🚀!

[yuanyuangoo]

Thank you so much.