Skip to content

Commit 5edf565

Browse files
kylesayrsdsikka
andauthored
[Bugfix][Tracing] Fix qwen2_5_vl (#1448)
## Purpose ## * Fix failing qwen_2_5_vl e2e test ## Changes ## * For recipes that run smoothquant before gptq, these recipes would populate `self.rope_deltas`. The existence of `self.rope_deltas` meant that subsequent tracing calls would not trigger [the condition to inject tracing metadata](https://github.com/vllm-project/llm-compressor/blob/main/src/llmcompressor/transformers/tracing/qwen2_5_vl.py#L406), leading to a tracing failure ## Testing ## * `CADENCE=weekly TEST_DATA_FILE=tests/lmeval/configs/vl_int8_w8a8_dynamic_per_token.yaml python3 -m pytest tests/lmeval/test_lmeval.py -sx` Signed-off-by: Kyle Sayers <[email protected]> Co-authored-by: Dipika Sikka <[email protected]>
1 parent 4b969af commit 5edf565

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

src/llmcompressor/transformers/tracing/qwen2_5_vl.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -413,8 +413,6 @@ def forward(
413413
second_per_grid_ts,
414414
attention_mask,
415415
)
416-
# TRACING: the position_ids shape is known
417-
position_ids = maybe_install_metadata_position_ids(position_ids, input_ids)
418416
self.rope_deltas = rope_deltas
419417
# then use the prev pre-calculated rope-deltas to get the correct position ids
420418
else:
@@ -431,6 +429,9 @@ def forward(
431429
position_ids = position_ids.add(delta)
432430
position_ids = position_ids.unsqueeze(0).expand(3, -1, -1)
433431

432+
# TRACING: the position_ids shape is known
433+
position_ids = maybe_install_metadata_position_ids(position_ids, input_ids)
434+
434435
outputs = self.model(
435436
input_ids=None,
436437
position_ids=position_ids,

0 commit comments

Comments
 (0)