Merging main changes as of 20250410 #15

Alexei-V-Ivanov-AMD · 2025-04-10T15:02:40Z

No description provided.

github-actions · 2025-04-10T15:02:53Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

Signed-off-by: Kebe <[email protected]>

Signed-off-by: 22quinn <[email protected]>

Signed-off-by: Matthew Bonanni <[email protected]>

…` and `LLM.embed` (#18800) Signed-off-by: DarkLight1337 <[email protected]>

Signed-off-by: Benji Beck <[email protected]> Co-authored-by: Cyrus Leung <[email protected]>

Signed-off-by: Isotr0py <[email protected]>

…ment (#23351) Signed-off-by: yewentao256 <[email protected]>

Signed-off-by: Arjun Reddy <[email protected]> Co-authored-by: Arjun Reddy <[email protected]>

…17149) Signed-off-by: Philip Chung <[email protected]>

Signed-off-by: jiabin.00 <[email protected]> Signed-off-by: Jee Jee Li <[email protected]> Co-authored-by: Jee Jee Li <[email protected]>

…odels (#23154) Signed-off-by: Chen Zhang <[email protected]>

…cator (#23079) Signed-off-by: sfeng33 <[email protected]>

…t is leveraged (#22667) Signed-off-by: Guillaume Calmettes <[email protected]>

Signed-off-by: Benji Beck <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Signed-off-by: jiang1.li <[email protected]>

Signed-off-by: Jee Jee Li <[email protected]>

Signed-off-by: Woosuk Kwon <[email protected]>

Signed-off-by: Russell Bryant <[email protected]>

Signed-off-by: carlory <[email protected]>

Signed-off-by: Chen Zhang <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Signed-off-by: Andy Xie <[email protected]>

…ts in the run-nightly-benchmark script (#23375) Signed-off-by: Naman Lalit <[email protected]>

Signed-off-by: Burkhard Ringlein <[email protected]>

…sion (#23353) Signed-off-by: Martin Hickey <[email protected]>

#23366) Signed-off-by: bppps <[email protected]> Co-authored-by: zouyu.zzx <[email protected]> Co-authored-by: bppps <[email protected]>

Signed-off-by: AzizCode92 <[email protected]> Signed-off-by: Michael Goin <[email protected]> Co-authored-by: Michael Goin <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

Signed-off-by: Yong Hoon Shin <[email protected]>

Signed-off-by: Didier Durand <[email protected]>

Signed-off-by: chzhang <[email protected]>

Signed-off-by: Lukas Geiger <[email protected]>

Signed-off-by: charlifu <[email protected]>

…ctness (#23854) Signed-off-by: Russell Bryant <[email protected]>

Signed-off-by: Isotr0py <[email protected]>

…ng on AMD (#23864) Signed-off-by: Jinghui Zhang <[email protected]>

Signed-off-by: wangxiyuan <[email protected]>

Signed-off-by: Jee Jee Li <[email protected]>

…ken can potentially throw in default vl mrope implementation. (#23895) Signed-off-by: Chenheli Hua <[email protected]>

Signed-off-by: zjy0516 <[email protected]> Signed-off-by: Jiangyun Zhu <[email protected]> Co-authored-by: Patrick von Platen <[email protected]>

Signed-off-by: Harry Mellor <[email protected]>

Signed-off-by: Woosuk Kwon <[email protected]> Signed-off-by: Max de Bayser <[email protected]> Co-authored-by: Woosuk Kwon <[email protected]>

Signed-off-by: jiang1.li <[email protected]>

Signed-off-by: wang.yuqi <[email protected]>

Signed-off-by: sfeng33 <[email protected]>

Signed-off-by: Huy Do <[email protected]> Co-authored-by: Michael Goin <[email protected]>

Signed-off-by: aditchawdhary <[email protected]>

Signed-off-by: EduardDurech <[email protected]> Co-authored-by: AllenHaoHuang <[email protected]>

Signed-off-by: Lukas Geiger <[email protected]>

Signed-off-by: Nick Hill <[email protected]>

…ag (#23942) Signed-off-by: vllmellm <[email protected]>

Signed-off-by: hongchao <[email protected]> Signed-off-by: Richard Zou <[email protected]> Co-authored-by: hongchao <[email protected]> Co-authored-by: Richard Zou <[email protected]> Co-authored-by: Richard Zou <[email protected]>

Signed-off-by: 22quinn <[email protected]> Co-authored-by: Cyrus Leung <[email protected]>

Signed-off-by: mgoin <[email protected]>

…ed models (#23824) Signed-off-by: Thomas Parnell <[email protected]>

Signed-off-by: Yong Hoon Shin <[email protected]>

Signed-off-by: Harry Mellor <[email protected]>

Signed-off-by: Roger Wang <[email protected]>

kebe7jun and others added 29 commits August 21, 2025 23:09

[Feature][Responses API] Support logprobs(non-stream) (#23319)

5368f76

Signed-off-by: Kebe <[email protected]>

[Core] Support custom executor qualname (#23314)

480bdf5

Signed-off-by: 22quinn <[email protected]>

[Kernel] Add FP8 support with FlashMLA backend (#22668)

19fe1a0

Signed-off-by: Matthew Bonanni <[email protected]>

[Deprecation] Remove prompt_token_ids arg fallback in `LLM.generate…

8896eb7

…` and `LLM.embed` (#18800) Signed-off-by: DarkLight1337 <[email protected]>

Migrate MllamaImagePixelInputs to TensorSchema (#22020)

0b9cc56

Signed-off-by: Benji Beck <[email protected]> Co-authored-by: Cyrus Leung <[email protected]>

[CI/Build] Skip Idefics3 and SmolVLM generation test again (#23356)

3ac8496

Signed-off-by: Isotr0py <[email protected]>

[Feature] Enable DeepGEMM Linear on B200; 1.5% E2E throughput improve…

394591e

…ment (#23351) Signed-off-by: yewentao256 <[email protected]>

[CI] Add end-to-end V1 min_tokens test coverage (#22495)

111692b

Signed-off-by: Arjun Reddy <[email protected]> Co-authored-by: Arjun Reddy <[email protected]>

[Misc] Add gemma3 chat template with pythonic-style function calling (#…

de9c085

…17149) Signed-off-by: Philip Chung <[email protected]>

[New Model] Add Seed-Oss model (#23241)

5964069

Signed-off-by: jiabin.00 <[email protected]> Signed-off-by: Jee Jee Li <[email protected]> Co-authored-by: Jee Jee Li <[email protected]>

[Attention] Refactor AttentionMetadata Preparation for Encoder-only M…

17373dc

…odels (#23154) Signed-off-by: Chen Zhang <[email protected]>

[P/D][Nixl] Make kv cache register compatible with hybrid memory allo…

5341565

…cator (#23079) Signed-off-by: sfeng33 <[email protected]>

[gpt-oss] add input/output usage in responses api when harmony contex…

0ba1b54

…t is leveraged (#22667) Signed-off-by: Guillaume Calmettes <[email protected]>

Migrate MiniCPMOAudioInputs to TensorSchema (#21847)

9987208

Signed-off-by: Benji Beck <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

[Bugfix] Fix pooling models on CPU backend (#23392)

88016c3

Signed-off-by: jiang1.li <[email protected]>

[V0 Deprecation] Remove V0 LoRA test (#23418)

285178b

Signed-off-by: Jee Jee Li <[email protected]>

[Misc] Move M-RoPE init logic to _init_mrope_positions (#23422)

808d2e9

Signed-off-by: Woosuk Kwon <[email protected]>

[Attention] Allow V1 flash_attn to support cross-attention (#23297)

281710e

Signed-off-by: Russell Bryant <[email protected]>

[misc] Remove outdate comment about runai_model_streamer (#23421)

695e7ad

Signed-off-by: carlory <[email protected]>

[Doc] Update the doc for log probs + prefix caching (#23399)

a073be6

Signed-off-by: Chen Zhang <[email protected]> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>

[Misc] local import code clean (#23420)

325aa3d

Signed-off-by: Andy Xie <[email protected]>

[Bug fix] Dynamically setting the backend variable for genai_perf_tes…

ebe1462

…ts in the run-nightly-benchmark script (#23375) Signed-off-by: Naman Lalit <[email protected]>

[Fix] Bump triton version in rocm-build requirements (#21630)

51a2153

Signed-off-by: Burkhard Ringlein <[email protected]>

[Bugfix]: Installing dev environment due to pydantic incompatible ver…

613a23b

…sion (#23353) Signed-off-by: Martin Hickey <[email protected]>

[Speculators][Speculative Decoding] Fix Qwen 2 Eagle3 Support (#23337)

88491c1

[BugFix] Fix the issue where image embeddings were incorrectly split.… (

424fb7a

#23366) Signed-off-by: bppps <[email protected]> Co-authored-by: zouyu.zzx <[email protected]> Co-authored-by: bppps <[email protected]>

Add unit tests for batched guided and non-guided requests (#23389)

b6d7d34

Signed-off-by: Yong Hoon Shin <[email protected]>

[Doc]: fix various typos in multiple files (#23179)

22cf679

Signed-off-by: Didier Durand <[email protected]>

WoosukKwon and others added 30 commits August 28, 2025 18:05

[V0 Deprecation] Remove V0 Samplers test (#23862)

b668055

[XPU] support data parallel for MoE models on XPU (#22887)

235c9db

Signed-off-by: chzhang <[email protected]>

[Models] Improve iteration over layers (#19497)

de533ab

Signed-off-by: Lukas Geiger <[email protected]>

[ROCm][Fix] Fix rocm build caused by #23791 (#23847)

006477e

Signed-off-by: charlifu <[email protected]>

[tests] Improve speed and reliability of test_transcription_api_corre…

c8b3b29

…ctness (#23854) Signed-off-by: Russell Bryant <[email protected]>

[Bugfix] Use ReplicatedLinear for SequenceClassification head (#23836)

98ac0cb

Signed-off-by: Isotr0py <[email protected]>

[BugFix][AMD][Deepseek] fix a dtype mismatch error for deepseek runni…

5264015

…ng on AMD (#23864) Signed-off-by: Jinghui Zhang <[email protected]>

[Platform] import activation_quant_fusion for CUDA only (#23882)

6597d7a

Signed-off-by: wangxiyuan <[email protected]>

Fix(async): Add support for truncate_prompt_tokens in AsyncLLM (#23800)

05d839c

[CI/Build] Clean up LoRA test (#23890)

b4f9e96

Signed-off-by: Jee Jee Li <[email protected]>

[mrope][Qwen2-VL] Fix edge case where getting index of image/video to…

2d0afcc

…ken can potentially throw in default vl mrope implementation. (#23895) Signed-off-by: Chenheli Hua <[email protected]>

[Misc] Fix warnings for mistral model (#23552)

885ca6d

Signed-off-by: zjy0516 <[email protected]> Signed-off-by: Jiangyun Zhu <[email protected]> Co-authored-by: Patrick von Platen <[email protected]>

Better errors for Transformers backend missing features (#23759)

934bebf

Signed-off-by: Harry Mellor <[email protected]>

[V0 Deprecation] Remove pooling model support in V0 (#23434)

2554b27

Signed-off-by: Woosuk Kwon <[email protected]> Signed-off-by: Max de Bayser <[email protected]> Co-authored-by: Woosuk Kwon <[email protected]>

[CPU] Enable data parallel for CPU backend (#23903)

ad39106

Signed-off-by: jiang1.li <[email protected]>

[Performance] V1 Classify Models E2E Performance Optimization (#23541)

d9e00db

Signed-off-by: wang.yuqi <[email protected]>

[Multimodal] Consolidate mm inputs into MultiModalFeatureSpec (#23779)

69f4635

Signed-off-by: sfeng33 <[email protected]>

Update PyTorch to 2.8.0 (#20358)

67c1490

Signed-off-by: Huy Do <[email protected]> Co-authored-by: Michael Goin <[email protected]>

Adds json_count_leaves utility function (#23899)

4f7cde7

Signed-off-by: aditchawdhary <[email protected]>

[MODEL] Apertus and XIELU (#23068)

1cf3753

Signed-off-by: EduardDurech <[email protected]> Co-authored-by: AllenHaoHuang <[email protected]>

[Models] Use in-place adds in Idefics2Vision (#23932)

0a2f4c0

Signed-off-by: Lukas Geiger <[email protected]>

[BugFix] Async scheduling and PP compatibility with DP (#23770)

d90d8eb

Signed-off-by: Nick Hill <[email protected]>

[CI] Add aiter to matching list of issue auto labeller for rocm t…

72a6913

…ag (#23942) Signed-off-by: vllmellm <[email protected]>

[RL][BugFix] Fix missing tokenizer error for token-in-token-out (#23904)

4d7fe40

Signed-off-by: 22quinn <[email protected]> Co-authored-by: Cyrus Leung <[email protected]>

Tuned H100/H200 triton fp8 block configs for fused_qkv_a_proj (#23939)

b7adf94

Signed-off-by: mgoin <[email protected]>

[Docs] [V1] [Hybrid] Add new documentation re: contributing mamba-bas…

1c26b42

…ed models (#23824) Signed-off-by: Thomas Parnell <[email protected]>

Revert gemma3n fast prefill changes (#23897)

8c3e199

Signed-off-by: Yong Hoon Shin <[email protected]>

[Misc] Make download_weights_from_hf more reliable (#23863)

5674a40

Signed-off-by: Harry Mellor <[email protected]>

[CI] Fix unavailable image remote URL (#23966)

d660c98

Signed-off-by: Roger Wang <[email protected]>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Merging main changes as of 20250410 #15

Merging main changes as of 20250410 #15

Uh oh!

Alexei-V-Ivanov-AMD commented Apr 10, 2025 •

edited by github-actions bot

Loading

Uh oh!

github-actions bot commented Apr 10, 2025

Uh oh!

Uh oh!

Merging main changes as of 20250410 #15

Are you sure you want to change the base?

Merging main changes as of 20250410 #15

Uh oh!

Conversation

Alexei-V-Ivanov-AMD commented Apr 10, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Apr 10, 2025

Uh oh!

Uh oh!

Alexei-V-Ivanov-AMD commented Apr 10, 2025 •

edited by github-actions bot

Loading