📚 Accumulate completions for logging #3217
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
lewtun
commented
Apr 2, 2025
| @@ -136,6 +136,9 @@ class GRPOConfig(TrainingArguments): | |||
| installed, it prints the sample. If `wandb` logging is enabled, it logs it to `wandb`. | |||
| num_completions_to_print (`int` or `None`, *optional*, defaults to `None`): | |||
| Number of completions to print with `rich`. If `None`, all completions are logged. | |||
| wandb_log_unique_prompts (`bool`, *optional*, defaults to `False`): | |||
qgallouedec
reviewed
Apr 2, 2025
Comment on lines
+907
to
+918
| # Log completions when we complete a full gradient accumulation cycle | ||
| # For logging across the gradient accumulation steps, we need to accumulate the data | ||
| is_last_step_in_grad_accum = ( | ||
| self._step % self.args.gradient_accumulation_steps == self.args.gradient_accumulation_steps - 1 | ||
| ) | ||
| should_log_completions = ( | ||
| self.log_completions | ||
| and self.state.global_step % self.args.logging_steps == 0 | ||
| and is_last_step_in_grad_accum | ||
| ) | ||
|
|
||
| # Collect data for logging throughout the accumulation steps |
There was a problem hiding this comment.
I think self.control.should_log achieves the same goal. Let me check
There was a problem hiding this comment.
So yes it achieves the same goal but this flag is only updated after the loss.backward so we can't use it here. I'm starting to think that it would make more sense to have all this code in self.log. I'm checking if it's possible.
There was a problem hiding this comment.
So yes It's possible, I'll do it in a follow-up PR
qgallouedec
approved these changes
Apr 2, 2025
qgallouedec
changed the title
yxliu-TAMU
pushed a commit
to mincheolseong/ECEN743-GRPO-Project-Proposal
that referenced
this pull request
Apr 20, 2025
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
What does this PR do?
This PR fixes a bug where the
richandwandbtables were not logging the complete set of unique prompts forgradient_accumulation_steps > 1. Previously what happened is that the tables we logged for each gradient accumulation step, which is confusing when inspecting the logs for unique prompts.Here's two runs with/without the fix for:
main: https://wandb.ai/huggingface/huggingface/runs/j3vwwb5t/workspace?nw=nwuserlewtunthis_pr: https://wandb.ai/huggingface/huggingface/runs/78q63hev/overviewThese runs have:
gradient_accumulation_steps=4num_generations=16per_device_train_batch_size=4num_gpus = 8so we expect
num_unique_prompts=8:We see that without the fix,
wandbonly displays 2 unique prompts in the table (i.e. once per gradient accumulation step). With the fix, the prompts are grouped together per step, which incidentally makes the table scrolling faster in the UI as we have fewer steps to query.Before submitting
Pull Request section?
to it if that's the case.
documentation guidelines.
Who can review?
Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.