Skip to content

[fully_async] fix: add fully async grpo qwen3-235b npu script in main branch#6012

Merged
wucong25 merged 4 commits intoverl-project:mainfrom
wangshuyang31:fully-async-235b-main
Apr 16, 2026
Merged

[fully_async] fix: add fully async grpo qwen3-235b npu script in main branch#6012
wucong25 merged 4 commits intoverl-project:mainfrom
wangshuyang31:fully-async-235b-main

Conversation

@wangshuyang31
Copy link
Copy Markdown
Contributor

What does this PR do?

Re-apply override_transformer_config in engine worker (partially roll back #5583), because that PR causes certain patches to not be applied properly.

Add fully async grpo qwen3-235b npu script

Checklist Before Starting

  • Search for similar PRs. Paste at least one query link here: ...
  • Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
    • {modules} include fsdp, megatron, veomni, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data, cfg, reward, fully_async, one_step_off
    • If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
    • {type} is in feat, fix, refactor, chore, test
    • If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
    • Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Test

For changes that can not be tested by CI (e.g., algorithm implementation, new model support), validate by experiment(s) and show results like training curve plots, evaluation results, etc.

API and Usage Example

Demonstrate how the API changes if any, and provide usage example(s) if possible.

# Add code snippet or script demonstrating how to use this

Design & Code Changes

Demonstrate the high-level design if this PR is complex, and list the specific changes.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Apr 15, 2026

CLA assistant check
All committers have signed the CLA.

Copy link
Copy Markdown
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a new shell script for running GRPO training with the Qwen3-235B model using Megatron on NPU hardware. It also adds support for NPU-specific MindSpeed patching in the engine workers by conditionally importing and applying a repatch function based on the transformer configuration. I have no feedback to provide.

@wuxibin89
Copy link
Copy Markdown
Collaborator

Duplicate with #5967

@wangshuyang31
Copy link
Copy Markdown
Contributor Author

Duplicate with #5967

thanks, I will check it out

@wangshuyang31 wangshuyang31 changed the title [fully_async] fix: re-apply override_transformer_config in engine worker & add fully async grpo qwen3-235b npu script in main branch [fully_async] fix: add fully async grpo qwen3-235b npu script in main branch Apr 16, 2026
@wucong25 wucong25 merged commit 369c544 into verl-project:main Apr 16, 2026
5 of 6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants