Skip to content

📢 Improve GRPO trainer error message for invalid num_generations #3199

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Apr 5, 2025
Merged

📢 Improve GRPO trainer error message for invalid num_generations #3199

Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c698cc4
Improve GRPO trainer error message for invalid num_generations
AliBakly Mar 31, 2025
e3656fe
Merge branch 'main' into fix-grpo-error-message
qgallouedec Apr 1, 2025
53672f3
Removed white space, format tests passed
AliBakly Apr 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions trl/trainer/grpo_trainer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -445,6 +445,11 @@ def data_collator(features): # No data collation is needed in GRPO
num_processes = self.accelerator.num_processes
global_batch_size = args.per_device_train_batch_size * num_processes
possible_values = [n_gen for n_gen in range(2, global_batch_size + 1) if (global_batch_size) % n_gen == 0]
if self.num_generations < 2:
raise ValueError(
f"GRPO requires at least 2 generations per prompt to calculate the advantages. "
f"You provided {self.num_generations}, which is less than the minimum required."
)
if self.num_generations not in possible_values:
raise ValueError(
f"The global train batch size ({num_processes} x {args.per_device_train_batch_size}) must be evenly "
Expand Down
Loading