Skip to content

Commit 0035c36

Browse files
arunmadhusudaireilly
authored andcommitted
[Example] Fix Qwen VL ignore list (vllm-project#1545)
**SUMMARY:** - Transformers v4.52 will map weights name for Qwen2-VL/Qwen2.5-VL: https://github.com/huggingface/transformers/blob/de4cf5a38e9678b9e465867a8a6b88ea727bea52/src/transformers/models/qwen2_vl/modeling_qwen2_vl.py#L1359-L1362 - So the qwen2-vl/qwen2.5-vl checkpoints saved after transformers v4.52 will have different weights name. - Update the ignore list to prevent the vision model from being quantized **CHANGES** - Add "model.visual." to ignore list **TEST PLAN:** Run [qwen2_vl_example.py](https://github.com/vllm-project/llm-compressor/blob/main/examples/multimodal_vision/qwen2_vl_example.py) Run [qwen_2_5_vl_example.py](https://github.com/vllm-project/llm-compressor/blob/main/examples/multimodal_vision/qwen_2_5_vl_example.py)
1 parent 36cc2e9 commit 0035c36

File tree

2 files changed

+2
-2
lines changed

2 files changed

+2
-2
lines changed

examples/multimodal_vision/qwen2_vl_example.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -83,7 +83,7 @@ def data_collator(batch):
8383
targets="Linear",
8484
scheme="W4A16",
8585
sequential_targets=["Qwen2VLDecoderLayer"],
86-
ignore=["lm_head", "re:visual.*"],
86+
ignore=["lm_head", "re:visual.*", "re:model.visual.*"],
8787
),
8888
]
8989

examples/multimodal_vision/qwen_2_5_vl_example.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ def data_collator(batch):
7777
targets="Linear",
7878
scheme="W4A16",
7979
sequential_targets=["Qwen2_5_VLDecoderLayer"],
80-
ignore=["lm_head", "re:visual.*"],
80+
ignore=["lm_head", "re:visual.*", "re:model.visual.*"],
8181
),
8282
]
8383

0 commit comments

Comments
 (0)