Skip to content

Commit 2994eff

Browse files
authored
fix get_vllm_engine bug (#463)
1 parent 85e54b6 commit 2994eff

File tree

2 files changed

+2
-2
lines changed

2 files changed

+2
-2
lines changed

docs/source/LLM/VLLM推理加速与部署.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ pip install -r requirements/llm.txt -U
2727
```
2828

2929
## 推理加速
30-
vllm不支持bnb和auto_gptq量化的模型. vllm支持的模型可以查看[支持的模型](./支持的模型和数据集.md#模型).
30+
vllm不支持bnb量化的模型. vllm支持的模型可以查看[支持的模型](./支持的模型和数据集.md#模型).
3131

3232
### qwen-7b-chat
3333
```python

swift/llm/utils/vllm_utils.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -26,7 +26,7 @@
2626
def get_vllm_engine(model_type: str,
2727
torch_dtype: Optional[Dtype] = None,
2828
*,
29-
model_id_or_path: Optional[None],
29+
model_id_or_path: Optional[str] = None,
3030
gpu_memory_utilization: float = 0.9,
3131
tensor_parallel_size: int = 1,
3232
max_model_len: Optional[int] = None,

0 commit comments

Comments
 (0)