FEAT: Decoupled CLIP ratio (DAPO Trick-I) #285
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
|
Hi @ZiyiTsang , thank you for the contribution! Could you please format the code with pre-commit? Please refer to the code quality section in the contribution guide: https://inclusionai.github.io/AReaL/contrib.html |
garrett4wade
reviewed
Sep 3, 2025
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
|
@ZiyiTsang hi ziyi, I don't know why this PR becomes wild... pre-commit should only format the files you changed, rather than all files inside the repo. Did you trigger something that formats all the code inside this repo? If possible, you can add me in your forked repo with write permissions. I'll push to this |
|
FYI I've made these changes in the |
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
|
This time ok finnally....emmm take mee looooog time. check this version |
garrett4wade
reviewed
Sep 3, 2025
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
ZiyiTsang
had a problem deploying
to
AReaL-unittests
GitHub Actions
Error
garrett4wade
approved these changes
Sep 3, 2025
garrett4wade
added a commit
that referenced
this pull request
Sep 5, 2025
Merge branch fw/refactor-examples of [email protected]:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/810?tab=comment Reviewed-by: 晓雷 <[email protected]> * FEAT: Decoupled CLIP ratio (DAPO Trick-I) (#285) * Add agent-related logging logic in ppo actor & Update notebook example (#290) * refactor examples
garrett4wade
added a commit
that referenced
this pull request
Sep 5, 2025
…pendent device meshes (#297) * PullRequest: 805 fix: remove `set_gradient_sync` call in fsdp engine to reduce memory usage Merge branch fw/fix-grad-sync of [email protected]:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/805 Reviewed-by: 晓雷 <[email protected]> * . * PullRequest: 810 refactor: remove unused code inside the examples folder Merge branch fw/refactor-examples of [email protected]:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/810?tab=comment Reviewed-by: 晓雷 <[email protected]> * FEAT: Decoupled CLIP ratio (DAPO Trick-I) (#285) * Add agent-related logging logic in ppo actor & Update notebook example (#290) * refactor examples * PullRequest: 778 Enhance FSDP and Ulysses integration with improved data parallel handling and cleanup Merge branch zwt/dp_head of [email protected]:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/778?tab=diff Reviewed-by: 博惟 <[email protected]> * Enhance FSDP and Ulysses integration with improved data parallel handling and cleanup * WIP * WIP * WIP * PullRequest: 814 [FIX] fix megatron grad sync for replicas Merge branch sxj/fix-megatron-grad-sync of [email protected]:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/814 Reviewed-by: 晓雷 <[email protected]> * [FIX] fix megatron grad sync for replicas * fix * fix --------- Co-authored-by: 巧林 <[email protected]> Co-authored-by: 冰临 <[email protected]>
29 tasks
mjbmjb
pushed a commit
to mjbmjb/AReaL
that referenced
this pull request
Sep 22, 2025
* FEAT: add CLIP_higher (DAPO Trick-I) * Change default value for eps_clip_higher * rewrite logic in fuctional (CLIP higher) * 你try to fromatting * try to fromatting * modify formula
mjbmjb
pushed a commit
to mjbmjb/AReaL
that referenced
this pull request
Sep 22, 2025
…pendent device meshes (inclusionAI#297) * PullRequest: 805 fix: remove `set_gradient_sync` call in fsdp engine to reduce memory usage Merge branch fw/fix-grad-sync of [email protected]:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/805 Reviewed-by: 晓雷 <[email protected]> * . * PullRequest: 810 refactor: remove unused code inside the examples folder Merge branch fw/refactor-examples of [email protected]:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/810?tab=comment Reviewed-by: 晓雷 <[email protected]> * FEAT: Decoupled CLIP ratio (DAPO Trick-I) (inclusionAI#285) * Add agent-related logging logic in ppo actor & Update notebook example (inclusionAI#290) * refactor examples * PullRequest: 778 Enhance FSDP and Ulysses integration with improved data parallel handling and cleanup Merge branch zwt/dp_head of [email protected]:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/778?tab=diff Reviewed-by: 博惟 <[email protected]> * Enhance FSDP and Ulysses integration with improved data parallel handling and cleanup * WIP * WIP * WIP * PullRequest: 814 [FIX] fix megatron grad sync for replicas Merge branch sxj/fix-megatron-grad-sync of [email protected]:inclusionAI/AReaL.git into gh https://code.alipay.com/inclusionAI/AReaL/pull_requests/814 Reviewed-by: 晓雷 <[email protected]> * [FIX] fix megatron grad sync for replicas * fix * fix --------- Co-authored-by: 巧林 <[email protected]> Co-authored-by: 冰临 <[email protected]>
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
In this PR, I decouple the CLIP ratio used to compute PPO loss, as one of the trick from DAPO.
To maximize availability, I did not change the
eps_clipkey. I only added eps_clip_higer and decoupled it when it has a value.Please review