Skip to content

Conversation

jiaqiw09
Copy link
Contributor

What does this PR do?

  • Add example scirpt to run mutip-turn grpo in vllm and fsdp
  • fix progressbar in dapo trainer
    • When enable_filter is enabled, DAPO runs multiple batch inferences before each actor update, but the progress bar advances once per inference—mismatching the true training step count and leading to confusion.

Checklist Before Starting

  • Search for similar PRs. Paste at least one query link here: ...
  • Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
    • {modules} include fsdp, megatron, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data
    • If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
    • {type} is in feat, fix, refactor, chore, test
    • If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
    • Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Test

For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluation results, etc.

API and Usage Example

Demonstrate how the API changes if any, and provide usage example(s) if possible.

# Add code snippet or script demonstrating how to use this

Design & Code Changes

Demonstrate the high-level design if this PR is complex, and list the specific changes.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new example script for multi-turn GRPO training using the vLLM backend and FSDP, and also includes a fix for a progress bar issue in the DAPO trainer. My review identifies critical issues in the new shell script where undefined variables are used, which would cause execution failures. I've provided comments to address these. The fix in the DAPO trainer appears correct and resolves the described problem of the progress bar advancing incorrectly.

@wuxibin89 wuxibin89 merged commit aa19c1a into volcengine:main Sep 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants