How to support Rest API calling in custom wrapper asynchronous model invoke function. #1562
Unanswered
sriramraja-krish
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
Hey I am building an custom wrapper for Model invoke logic, So in my case I am using rest api, I am changing the below code into rest api, async def a_generate(
self, prompt: str, schema: Optional[BaseModel] = None
) -> str:
"""Asynchronously generates text from a prompt.
I am trying above function to change as below,
async with aiohttp.ClientSession() as session:
async with session.post(url, headers=headers, json=payload) as resp:
if resp.status != 200:
raise Exception(f"Request failed with status {resp.status}")
data = await resp.json()
But the problem is the prompt has not include previous opinions , So the request passed without context, I tried asyn call for http request still no use , when i try run_asynx =False , It works for first metric, but not for other metric , how to achieve this ,
Beta Was this translation helpful? Give feedback.
All reactions