[Bugfix] Allow shared_experts skip quantization for DeepSeekV2/V3 (#14100)

mgoin · web-flow · commit 2b04c209ee98 · 2025-03-03T14:20:24.000-07:00
Signed-off-by: mgoin &lt;mgoin64@gmail.com&gt;
diff --git a/vllm/model_executor/models/deepseek_v2.py b/vllm/model_executor/models/deepseek_v2.py
@@ -145,6 +145,7 @@ def __init__(
                 hidden_act=config.hidden_act,
                 quant_config=quant_config,
                 reduce_results=False,
+                prefix=f"{prefix}.shared_experts",
             )
 
     def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:

Original file line number	Diff line number	Diff line change
`@@ -145,6 +145,7 @@ def __init__(`
`145`	`145`	`hidden_act=config.hidden_act,`
`146`	`146`	`quant_config=quant_config,`
`147`	`147`	`reduce_results=False,`
	`148`	`+ prefix=f"{prefix}.shared_experts",`
`148`	`149`	`)`
`149`	`150`
`150`	`151`	`def forward(self, hidden_states: torch.Tensor) -> torch.Tensor:`