[MM Encoder] General encoder performance improvement #23884
Description
🚀 The feature, motivation and pitch
On vLLM, most multimodal model support was directly contributed by the model vendors, but sometimes the implementation can still be improved in terms of inference performance (e.g, by leveraging fused operations or avoid cumemcpy).
Two examples are
- Remove unnecessary CUDA sync of qwen image and video preprocess #22792
- [Model] Switch to Fused RMS norm in Qwen2.5_VL model. #22184
We should more actively hunt for such improvement, in particular for more popular models such as Qwen2.5VL, InternVL, GLM4.5V, etc. It should be also made sure that such improvement does not result in accuracy regression.
Alternatives
No response
Additional context
No response
Before submitting a new issue...
- Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.