Skip to content

Commit a3968c7

Browse files
authored
fix lmdeploy & vlm (#1530)
1 parent 942aa86 commit a3968c7

18 files changed

+1025
-825
lines changed

docs/source/LLM/LmDeploy推理加速与部署.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
# LmDeploy推理加速与部署
2+
支持lmdeploy推理加速的模型可以查看[支持的模型](支持的模型和数据集.md#模型).
23

34
## 目录
45
- [环境准备](#环境准备)

docs/source/LLM/VLLM推理加速与部署.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11

22
# VLLM推理加速与部署
3+
vllm支持的模型可以查看[支持的模型](支持的模型和数据集.md#模型).
34

45
## 目录
56
- [环境准备](#环境准备)
@@ -27,7 +28,6 @@ pip install -r requirements/llm.txt -U
2728
```
2829

2930
## 推理加速
30-
vllm支持的模型可以查看[支持的模型](支持的模型和数据集.md#模型).
3131

3232
### 使用python
3333
```python

docs/source/LLM/支持的模型和数据集.md

Lines changed: 379 additions & 379 deletions
Large diffs are not rendered by default.

docs/source/Multi-Modal/LmDeploy推理加速文档.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
# LmDeploy推理加速与部署
2+
支持lmdeploy推理加速的多模态模型可以查看[支持的模型](../LLM/支持的模型和数据集.md#多模态大模型).
23

34
## 目录
45
- [环境准备](#环境准备)

docs/source/Multi-Modal/vLLM推理加速文档.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
# vLLM推理加速文档
2-
ms-swift已接入了vLLM对多模态模型进行推理加速. 支持的模型可以查看[支持的模型和数据集](../LLM/支持的模型和数据集.md#多模态大模型). 需要注意点是,使用vLLM进行加速会对推理效果产生略微影响, 请确保可以忍受这种损失来提升推理速度.
2+
ms-swift已接入了vLLM对多模态模型进行推理加速. 支持的模型可以查看[支持的模型和数据集](../LLM/支持的模型和数据集.md#多模态大模型).
33

44
## 目录
55
- [环境准备](#环境准备)

docs/source_en/LLM/LmDeploy-inference-acceleration-and-deployment.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
# LmDeploy Inference Acceleration and Deployment
2+
Models that support inference acceleration using lmdeploy can be found at [Supported Models](Supported-models-datasets.md#LLM).
23

34
## Table of Contents
45
- [Environment Preparation](#environment-preparation)

docs/source_en/LLM/Supported-models-datasets.md

Lines changed: 379 additions & 379 deletions
Large diffs are not rendered by default.

docs/source_en/LLM/VLLM-inference-acceleration-and-deployment.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
# VLLM Inference Acceleration and Deployment
2+
The models supported by vllm can be found in [Supported Models](Supported-models-datasets.md#Models).
23

34
## Table of Contents
45
- [Environment Preparation](#environment-preparation)
@@ -24,7 +25,6 @@ pip install -r requirements/llm.txt -U
2425
```
2526

2627
## Inference Acceleration
27-
The models supported by vllm can be found in [Supported Models](Supported-models-datasets.md#Models).
2828

2929
### Using Python
3030
```python

docs/source_en/Multi-Modal/LmDeploy-inference-acceleration.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
# LmDeploy Inference Acceleration and Deployment
2+
MLLM that support inference acceleration using lmdeploy can be found at [Supported Models](../LLM/Supported-models-datasets.md#MLLM).
23

34
## Table of Contents
45
- [Environment Preparation](#environment-preparation)

docs/source_en/Multi-Modal/vllm-inference-acceleration.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
# vLLM Inference Acceleration Documentation
22

3-
ms-swift has integrated vLLM for accelerating inference of multimodal models. Check out the supported models in [Supported Models and Datasets Documentation](../LLM/Supported-models-datasets.md). It's worth noting that using vLLM for acceleration may have a slight impact on the inference quality, so please ensure you can tolerate this loss in order to boost the inference speed.
3+
ms-swift has integrated vLLM for accelerating inference of multimodal models. Check out the supported models in [Supported Models and Datasets Documentation](../LLM/Supported-models-datasets.md#MLLM).
44

55
## Table of Contents
66
- [Environment Setup](#environment-setup)

0 commit comments

Comments
 (0)