Skip to content

Rebase 2025.01.28 - attempt 2 #750

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 109 commits into from
Jan 29, 2025
Merged

Rebase 2025.01.28 - attempt 2 #750

merged 109 commits into from
Jan 29, 2025

Conversation

@kzawora-intel
Copy link

@kzawora-intel kzawora-intel commented Jan 29, 2025

No description provided.

youkaichao and others added 30 commits January 21, 2025 19:32
@youkaichao
[torch.compile] transparent compilation with more logging (vllm-proje…
c81081f 
…ct#12246)

Signed-off-by: youkaichao <[email protected]>
@ywang96
[V1][Bugfix] Fix data item ordering in mixed-modality inference (vllm…
b197a5c 
…-project#12259)

Signed-off-by: Roger Wang <[email protected]>
@tdoublep
Remove pytorch comments for outlines + compressed-tensors (vllm-proje…
9a7c3a0 
…ct#12260)

Signed-off-by: Thomas Parnell <[email protected]>
@MengqingCao
[Platform] improve platforms getattr (vllm-project#12264)
c646128 
Signed-off-by: Mengqing Cao <[email protected]>
@youkaichao
[ci/build] update nightly torch for gh200 test (vllm-project#12270)
3aec49e 
Signed-off-by: youkaichao <[email protected]>
@joennlae
[Bugfix] fix race condition that leads to wrong order of token return…
9705b90 
…ed (vllm-project#10802)

Signed-off-by: Jannis Schönleber <[email protected]>
@jinzhen-lin
[Kernel] fix moe_align_block_size error condition (vllm-project#12239)
1e60f87 
Signed-off-by: Jinzhen Lin <[email protected]>
@rickyyx
[v1][stats][1/n] Add RequestStatsUpdate and RequestStats types (vllm-…
132a132 
…project#10907)

Signed-off-by: rickyx <[email protected]>
@andylolu2
[Bugfix] Multi-sequence broken (vllm-project#11898)
18fd4a8 
Signed-off-by: Andy Lo <[email protected]>
@codefromthecrypt
[Misc] Remove experimental dep from tracing.py (vllm-project#12007)
347eeeb 
Signed-off-by: Adrian Cole <[email protected]>
@wangxiyuan
[Misc] Set default backend to SDPA for get_vit_attn_backend (vllm-pro…
fa9ee08 
…ject#12235)

Signed-off-by: wangxiyuan <[email protected]>
@janimo
[Core] Free CPU pinned memory on environment cleanup (vllm-project#10477
9c485d9 
)
@divakar-amd
[bugfix] moe tuning. rm is_navi() (vllm-project#12273)
2acba47 
Signed-off-by: Divakar Verma <[email protected]>
@maleksan85
[BUGFIX] When skip_tokenize_init and multistep are set, execution cra…
69196a9 
…shes (vllm-project#12277)

Signed-off-by: maleksan85 <[email protected]>
Co-authored-by: maleksan85 <[email protected]>
@hongxiayang
[Documentation][AMD] Add information about prebuilt ROCm vLLM docker …
09ccc9c 
…for perf validation purpose (vllm-project#12281)

Signed-off-by: Hongxia Yang <[email protected]>
@DarkLight1337
[VLM] Simplify post-processing of replacement info (vllm-project#12269)
df76e5a 
Signed-off-by: DarkLight1337 <[email protected]>
@khluu
[ci/lint] Add back default arg for pre-commit (vllm-project#12279)
64ea24d 
Signed-off-by: kevin <[email protected]>
@liangfu
[CI] add docker volume prune to neuron CI (vllm-project#12291)
016e367 
Signed-off-by: Liangfu Chen <[email protected]>
@DarkLight1337
[Ci/Build] Fix mypy errors on main (vllm-project#12296)
cbdc4ad 
Signed-off-by: DarkLight1337 <[email protected]>
@njhill
[Benchmark] More accurate TPOT calc in benchmark_serving.py (vllm-p…
222a9dc 
…roject#12288)

Signed-off-by: Nick Hill <[email protected]>
@youkaichao
[core] separate builder init and builder prepare for each batch (vllm…
66818e5 
…-project#12253)

Signed-off-by: youkaichao <[email protected]>
@MengqingCao
[Build] update requirements of no-device (vllm-project#12299)
4004f14 
Signed-off-by: Mengqing Cao <[email protected]>
@youkaichao
[Core] Support fully transparent sleep mode (vllm-project#11743)
68ad4e3 
Signed-off-by: youkaichao <[email protected]>
@DarkLight1337
[VLM] Avoid unnecessary tokenization (vllm-project#12310)
cd7b6f0 
Signed-off-by: DarkLight1337 <[email protected]>
@xffxff
[Model][Bugfix]: correct Aria model output (vllm-project#12309)
528dbca 
Signed-off-by: xffxff <[email protected]>
@ywang96
[Bugfix][VLM] Fix mixed-modality inference backward compatibility for…
16366ee 
… V0 (vllm-project#12313)

Signed-off-by: Roger Wang <[email protected]>
@DarkLight1337
[Doc] Add docs for prompt replacement (vllm-project#12318)
6609cdf 
Signed-off-by: DarkLight1337 <[email protected]>
@WangErXiao
[Misc] Fix the error in the tip for the --lora-modules parameter (vll…
fc66dee 
…m-project#12319)

Signed-off-by: wangerxiao <[email protected]>
@jeejeelee
[Misc] Improve the readability of BNB error messages (vllm-project#12320
84bee4b 
)

Signed-off-by: Jee Jee Li <[email protected]>
@kzawora-intel
[Bugfix] Fix HPU multiprocessing executor (vllm-project#12167)
96f6a75 
Signed-off-by: Konrad Zawora <[email protected]>
WoosukKwon and others added 22 commits January 27, 2025 01:09
@WoosukKwon
[V1][Minor] Minor optimizations for update_from_output (vllm-project#…
624a1e4 
…12454)

Signed-off-by: Woosuk Kwon <[email protected]>
@Isotr0py
[Bugfix] Fix gpt2 GGUF inference (vllm-project#12467)
ce69f7f 
Signed-off-by: Isotr0py <[email protected]>
@LucasWilkinson
[Build] Only build 9.0a for scaled_mm and sparse kernels (vllm-projec…
103bd17 
…t#12339)

Signed-off-by: Lucas Wilkinson <[email protected]>
@markmc
[V1][Metrics] Add initial Prometheus logger (vllm-project#12416)
01ba927 
Signed-off-by: Mark McLoughlin <[email protected]>
@WoosukKwon
[V1][CI/Test] Do basic test for top-p & top-k sampling (vllm-project#…
3f1fc74 
…12469)

Signed-off-by: Woosuk Kwon <[email protected]>
@abmfy @youkaichao
[FlashInfer] Upgrade to 0.2.0 (vllm-project#11194)
2bc3fbb 
Signed-off-by: Bowen Wang <[email protected]>
Signed-off-by: youkaichao <[email protected]>
Co-authored-by: youkaichao <[email protected]>
@NickLucche @wallashss
[Feature] [Spec decode]: Enable MLPSpeculator/Medusa and `prompt_logp…
6116ca8 
…robs` with ChunkedPrefill (vllm-project#10132)

Signed-off-by: NickLucche <[email protected]>
Signed-off-by: wallashss <[email protected]>
Co-authored-by: wallashss <[email protected]>
@hmellor
Update pre-commit hooks (vllm-project#12475)
823ab79 
Signed-off-by: Harry Mellor <[email protected]>
@liangfu @JF-D
[Neuron][Kernel] NKI-based flash-attention kernel with paged KV cache (…
ddee88d 
…vllm-project#11277)

Signed-off-by: Liangfu Chen <[email protected]>
Co-authored-by: Jiangfei Duan <[email protected]>
@mgoin
Fix bad path in prometheus example (vllm-project#12481)
426a5c3 
Signed-off-by: mgoin <[email protected]>
@hosseinsarshar
[CI/Build] Fixed the xla nightly issue report in vllm-project#12451 (v…
23a7cbc 
…llm-project#12453)
@gmarinho2
[FEATURE] Enables offline /score for embedding models (vllm-project#1…
0f465ab 
…2021)

Signed-off-by: Gabriel Marinho <[email protected]>
@MengqingCao
[CI] fix pre-commit error (vllm-project#12494)
dd66fd2 
Signed-off-by: Mengqing Cao <[email protected]>
@ywang96
Update README.md with V1 alpha release (vllm-project#12495)
8cbc424
@robertgshaw2-redhat
[V1] Include Engine Version in Logs (vllm-project#12496)
e29d435 
Signed-off-by: [email protected] <[email protected]>
@schoennenbeck
[Core] Make raw_request optional in ServingCompletion (vllm-project#1…
2079e43 
…2503)

Signed-off-by: Sebastian Schönnenbeck <[email protected]>
@kzawora-intel
Merge remote-tracking branch 'upstream/main' into HEAD
7140ecc
@kzawora-intel
fix kv scales comparison
4c0e3de
@kzawora-intel
moe fixes
f3aabba
@kzawora-intel
spec decode fixes
54cfccf
@kzawora-intel
make pre-commit happy
32e6c5e
@kzawora-intel
skip spec decode tests w/ prefill chunking on hpu
18f03cd
@kzawora-intel kzawora-intel merged commit 73afddb into habana_main Jan 29, 2025
31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@michalkuligowski michalkuligowski michalkuligowski approved these changes

@madamczyk-intel madamczyk-intel Awaiting requested review from madamczyk-intel madamczyk-intel is a code owner

@mgawarkiewicz mgawarkiewicz Awaiting requested review from mgawarkiewicz

@vivekgoe vivekgoe Awaiting requested review from vivekgoe vivekgoe is a code owner

@afierka-intel afierka-intel Awaiting requested review from afierka-intel afierka-intel is a code owner

Assignees

No one assigned

Labels

rebase

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.