-
Notifications
You must be signed in to change notification settings - Fork 21
♻️ fix vllm:main for model_config.task #341
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
👋 Hi! Thank you for contributing to vLLM support on Spyre.
Or this can be done with
Now you are good to go 🚀 |
b47473c
to
2ab92e5
Compare
01e3d37
to
cce356b
Compare
Signed-off-by: Prashant Gupta <[email protected]>
Signed-off-by: Prashant Gupta <[email protected]>
Signed-off-by: Prashant Gupta <[email protected]>
Signed-off-by: Prashant Gupta <[email protected]>
Signed-off-by: Prashant Gupta <[email protected]>
Signed-off-by: Prashant Gupta <[email protected]>
Signed-off-by: Prashant Gupta <[email protected]>
Signed-off-by: Prashant Gupta <[email protected]>
Signed-off-by: Prashant Gupta <[email protected]>
Signed-off-by: Prashant Gupta <[email protected]>
Signed-off-by: Prashant Gupta <[email protected]>
Co-authored-by: Max de Bayser <[email protected]> Signed-off-by: Max de Bayser <[email protected]> Signed-off-by: Prashant Gupta <[email protected]>
Signed-off-by: Prashant Gupta <[email protected]>
c2e428f
to
284b66c
Compare
Signed-off-by: Prashant Gupta <[email protected]>
bot:test |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
It seems
model_config.task
is deprecated, instead from what I understand we can usemodel_config.supported_tasks
, which is initialized when an llm engine is instantiated:https://github.com/vllm-project/vllm/pull/21470/files#diff-7eaad0b7dee0626bf29d10081b0f0c5e3ea15a4af97e7b182a4e0d35f8346953R705-R706
To maintain backward compatibility, it's a bit tricky since:
model_config.task
pointing to the task andmodel_config.supported_tasks
as a list of all tasks which could contain more than 1 taskmain
now populatesmodel_config.supported_tasks
as the only task the model supports.