How to turn-off "enable_thinking" using "request" in qwen3? 如何在qwen3使用request调用API时禁用思考模型? #1343
timothy-WangS
started this conversation in
Ideas
Replies: 3 comments 1 reply
-
|
我也遇到了同样的问题 |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
not supported by ollama. |
Beta Was this translation helpful? Give feedback.
1 reply
-
|
提示词结尾加上 /no_think 指令。 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
我在使用ollama部署qwen3时使用api调用,但是设置“enable_thinking=False”会出现报错(如下)
然而实际的输出大部分时候是没有部分内容的(qwen3:8b)。想知道大家是怎么设置enable_thinking这个参数的,或者能不能用request-options调整这个参数?还是只能在prompt里面要求禁用思考?
代码:
conf.OLLAMA_API_BASE_URL、conf.OLLAMA_API_PORT是ollama的地址和端口
model_name="qwen3:8b"
url = f"{conf.OLLAMA_API_BASE_URL}:{conf.OLLAMA_API_PORT}/api/generate"
payload = {
'stream': False,
'model': model_name,
'prompt': my_prompt,
'options': {
'temperature': temperature,
省略其他部分 ...
'enable_thinking': False
}
}
报错:level=WARN source=types.go:570 msg="invalid option provided" option=enable_thinking
目前查到可能是思路的两种方法:
1、在prompt前加上/no_think
eg: prompt="你好" --> prompt="/no_think 你好"
2、在vllm中使用"extra_body" (也许可以这样用在request?)
url = f"{conf.OLLAMA_API_BASE_URL}:{conf.OLLAMA_API_PORT}/api/generate"
payload = {
'stream': False,
'model': model_name,
'prompt': my_prompt,
'options': {
'temperature': temperature,
省略其他部分 ...
"extra_body": {
"chat_template_kwargs": {
"enable_thinking": False,
}
},
}
}
但这种会报错level=WARN source=types.go:570 msg="invalid option provided" option=extra_body
因为部署任务对响应时间要求比较高,我比较倾向于在payload的option中进行处理,或者其他能确保屏蔽思维链的方式,有没有比较推荐的做法?
Beta Was this translation helpful? Give feedback.
All reactions