inaccurate eval loss computation

[wwt17]

transformers/examples/pytorch/language-modeling/run_clm_no_trainer.py

Line 657 in 1f0b490

losses.append(accelerator.gather_for_metrics(loss.repeat(args.per_device_eval_batch_size)))

The last batch may be smaller, since drop_last default to False for DataLoader construction. Thus, computing the mean over the losses is inaccurate.
Same issue for the train loss:

transformers/examples/pytorch/language-modeling/run_clm_no_trainer.py

Line 673 in 1f0b490

"train_loss": total_loss.item() / len(train_dataloader),