[Feature]: support voice llm like cosyvoice

[balcklive]

🚀 The feature, motivation and pitch

As we can see that vllm already support many vision language models. Do we have a plan to support voice models like cosyVoice?

Alternatives

No response

Additional context

No response

[ywang96]

I took a look at the architecture of CosyVoice and it looks a bit more complicated than what we can support today (mostly because of the flow matching).

However, we should be able to support audio-LLM with simple audio encoder + LM architecture (such as Qwen-audio) pretty easily given our current infrastructure for multi-modality. If you're interested, please feel free to open a PR!