Skip to content

Conversation

ccclyu
Copy link
Collaborator

@ccclyu ccclyu commented Oct 3, 2025

What does this PR do?

The model has been re-assigned to its inner model model.model so it does not have layers . fixed the reassign issue and refactor the code logic.

model = getattr(model, "model", None) or getattr(model, "language_model", None)
if model is None:
raise ValueError("The provided model does not have a valid 'model' or 'language_model' attribute.")
for layer in model.layers:

Checklist Before Starting

  • Search for similar PRs. Paste at least one query link here: Current MLP_ATTR_MAPPING dictionary is wrong. #2834
  • Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
    • {modules} include fsdp, megatron, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data
    • If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
    • {type} is in feat, fix, refactor, chore, test
    • If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
    • Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Design & Code Changes

Demonstrate the high-level design if this PR is complex, and list the specific changes.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly fixes a bug where type(model) was being used after the model variable had been reassigned to an inner model, causing issues with attribute lookups. The fix involves storing the original model type before this reassignment. The changes also include beneficial refactoring that improves code robustness and efficiency. My review includes one high-severity comment to address a potential issue where subclasses of supported models might not be handled correctly, with a suggestion to use the method resolution order (MRO) for a more robust implementation.

@ccclyu ccclyu changed the title [misc] fix model reassign to inner model in vllm patch file [misc, fix] fix model reassign to inner model in vllm patch file Oct 3, 2025
@ccclyu ccclyu changed the title [misc, fix] fix model reassign to inner model in vllm patch file [misc] fix: model reassign to inner model in vllm patch file Oct 3, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant