ValueError: Attempting to unscale FP16 gradients.

[MilesQLi]

The command is
~/repos/Open-Assistant/model/model_training$ python trainer_sft.py --configs galactica-125m webgpt_dataset_only

Python 3.10, libraries installed as instructed.

error_log.txt

[jeangeezhang]

I got same issue
Python == 3.10
peft == 0.3.0
torch == 1.13.1

[andreaskoepf]

Please try adding dtype: float32 to the model configuration.

[theophilegervet]

@MilesQLi @jeangeezhang I'm facing the same issue. Were you able to fix it?

[bhauptvogel]

Is it not possible to train the model with lower precision? e.g. dtype: float16