TypeError: unsupported operand type(s) for *: 'IndexedSlices' and 'float'

[353xiong]

e_w = gradients[i] * self.rho / (grad_norm + 1e-12)

TypeError: unsupported operand type(s) for *: 'IndexedSlices' and 'float'

How to deal with it?

[Jannoshh]

It seems like the variable 'gradients' is not a list (which it should be).
If you are trying to minimize a single tensor using a GradientTape like the following:

try
index_slices = tape.gradient(r, [v])
instead of
index_slices = tape.gradient(r, v)

If you are trying something different, could you give me some more details?

[dennism-tulcolabs]

I ran into the same error (TF 2.5), and replacing

e_w = gradients[i] * rho / (grad_norm + eps)

with

e_w = tf.math.scalar_mul(rho, gradients[i]) / (grad_norm + eps)

appears to fix it.