multi-LoRA as extra models in OpenAI server#2775

Merged

Yard1 merged 31 commits intovllm-project:mainfrom

jvmncs:openai-lora

Feb 17, 2024

Commits on Feb 5, 2024

use lora request in OpenAIServiing; expose lora modules as parameter in openai server entrypoint
jvmncs
committed
add single completion test cases for lora adapters
jvmncs
committed

Commits on Feb 6, 2024

Commits on Feb 14, 2024

[Minor] Fix benchmark_latency script (#2765)

WoosukKwon
authored and
jvmncs
committed
[ROCm] Fix some kernels failed unit tests (#2498)

hongxiayang
authored and
jvmncs
committed
Set local logging level via env variable (#2774)

gardberg
authored and
jvmncs
committed
[ROCm] Fixup arch checks for ROCM (#2627)

dllehr-amd
authored and
jvmncs
committed
Add fused top-K softmax kernel for MoE (#2769)

WoosukKwon
authored and
jvmncs
committed
modelscope: fix issue when model parameter is not a model id but path of the model. (#2489)

liuyhwangyh
authored and
jvmncs
committed
[Minor] More fix of test_cache.py CI test failure (#2750)

LiuXiaoxuanPKU
authored and
jvmncs
committed
[ROCm] Fix build problem resulted from previous commit related to FP8 kv-cache support (#2790)

hongxiayang
authored and
jvmncs
committed
Add documentation on how to do incremental builds (#2796)

pcmoritz
authored and
jvmncs
committed
[Ray] Integration compiled DAG off by default (#2471)

rkooo567
authored and
jvmncs
committed
Disable custom all reduce by default (#2808)

WoosukKwon
authored and
jvmncs
committed
[ROCm] support Radeon™ 7900 series (gfx1100) without using flash-attention (#2768)

hongxiayang
authored and
jvmncs
committed
Add documentation section about LoRA (#2834)

pcmoritz
authored and
jvmncs
committed
Refactor 2 awq gemm kernels into m16nXk32 (#2723)

authored and
jvmncs
committed
Serving Benchmark Refactoring (#2433)

ywang96
authored and
jvmncs
committed
[CI] Ensure documentation build is checked in CI (#2842)

simon-mo
authored and
jvmncs
committed
Refactor llama family models (#2637)

esmeetu
authored and
jvmncs
committed
Revert "Refactor llama family models (#2637)" (#2851)

pcmoritz
authored and
jvmncs
committed
Use CuPy for CUDA graphs (#2811)

WoosukKwon
authored and
jvmncs
committed
Remove Yi model definition, please use `LlamaForCausalLM` instead (#2854)

authored and
jvmncs
committed
Add LoRA support for Mixtral (#2831)

tterrysun
authored and
jvmncs
committed
Migrate InternLMForCausalLM to LlamaForCausalLM (#2860)

authored and
jvmncs
committed
Fix internlm after https://github.com/vllm-project/vllm/pull/2860 (#2861)

pcmoritz
authored and
jvmncs
committed
[Fix] Fix memory profiling when GPU is used by multiple processes (#2863)

WoosukKwon
authored and
jvmncs
committed
append lora serving instructions to lora documentation
jvmncs
committed
Merge branch 'main' into openai-lora
jvmncs
authored