Update AMP and DDP usage for imagenet main_amp.py

[tinglvv]

cc @Aidyn-A

[ptrblck]

Where would this be used?

[tinglvv]

seems used here

apex/examples/imagenet/main_amp.py

Lines 363 to 366 in 7ae82aa

	# to_python_float incurs a host<->device sync
	losses.update(to_python_float(reduced_loss), input.size(0))
	top1.update(to_python_float(prec1), input.size(0))
	top5.update(to_python_float(prec5), input.size(0))

[tinglvv]

Weird it was never defined previously

[tinglvv]

These lines were added 6 years ago, my guess is the function definition was removed in between?
The test that is related to this main_amp.py may have been failed for more than 3 years, I saw a bug to deprecate the usage of DDP/AMP in apex that was opened on 2022..

[Aidyn-A]

It was defined in the removed code for AMP:

apex/apex/fp16_utils/fp16util.py

Lines 178 to 182 in e13873d

	def to_python_float(t):
	if hasattr(t, 'item'):
	return t.item()
	else:
	return t[0]

(another reason why doing something like from apex.fp16_utils import * is undesirable)

[Aidyn-A]

These changes look good to me. But we will need to come up with something new in future.