Skip to content

Add OpenVINO GenAI backend #607

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: main
Choose a base branch
from
Open

Add OpenVINO GenAI backend #607

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
22da129
Update openvino.py
openvino-dev-samples Mar 6, 2025
bb52139
Merge branch 'QwenLM:main' into main
openvino-dev-samples Jun 10, 2025
70d01ea
add openvino genai
openvino-dev-samples Jun 10, 2025
1a9a3c7
update ov genai chat mode trigger
openvino-dev-samples Jun 10, 2025
cae755a
update full prompt trigger
openvino-dev-samples Jun 10, 2025
616381a
remove stop token
openvino-dev-samples Jun 11, 2025
6c19411
fix issue for no stream mode
openvino-dev-samples Jun 11, 2025
4a50d10
rebase
openvino-dev-samples Jun 12, 2025
c17bfa1
add genai template set
openvino-dev-samples Jun 12, 2025
c09e8b8
optimize seed deleting
openvino-dev-samples Jun 12, 2025
a0fea31
add WA for 4B model
openvino-dev-samples Jun 13, 2025
cb74976
add disable thinking for native tokenizer
openvino-dev-samples Jun 15, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion qwen_agent/llm/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
from .azure import TextChatAtAzure
from .base import LLM_REGISTRY, BaseChatModel, ModelServiceError
from .oai import TextChatAtOAI
from .openvino import OpenVINO
from .openvino import OpenVINO, OpenVINOGenAI
from .transformers_llm import Transformers
from .qwen_dashscope import QwenChatAtDS
from .qwenaudio_dashscope import QwenAudioChatAtDS
Expand Down Expand Up @@ -109,6 +109,7 @@ def get_chat_model(cfg: Union[dict, str] = 'qwen-plus') -> BaseChatModel:
'QwenAudioChatAtDS',
'QwenOmniChatAtOAI',
'OpenVINO',
'OpenVINOGenAI',
'Transformers',
'get_chat_model',
'ModelServiceError',
Expand Down
Loading