We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
There was an error while loading. Please reload this page.
1 parent 85e54b6 commit 2994effCopy full SHA for 2994eff
docs/source/LLM/VLLM推理加速与部署.md
@@ -27,7 +27,7 @@ pip install -r requirements/llm.txt -U
27
```
28
29
## 推理加速
30
-vllm不支持bnb和auto_gptq量化的模型. vllm支持的模型可以查看[支持的模型](./支持的模型和数据集.md#模型).
+vllm不支持bnb量化的模型. vllm支持的模型可以查看[支持的模型](./支持的模型和数据集.md#模型).
31
32
### qwen-7b-chat
33
```python
swift/llm/utils/vllm_utils.py
@@ -26,7 +26,7 @@
26
def get_vllm_engine(model_type: str,
torch_dtype: Optional[Dtype] = None,
*,
- model_id_or_path: Optional[None],
+ model_id_or_path: Optional[str] = None,
gpu_memory_utilization: float = 0.9,
tensor_parallel_size: int = 1,
max_model_len: Optional[int] = None,
0 commit comments