improvement: updated ai21 integration #1228
Conversation
|
DescriptionSummary By MatterAI
🔄 What ChangedThis Pull Request updates the integration with the AI21 Chat Completions API. Key changes include:
🔍 Impact of the ChangeThese changes enable the system to send more complex, multi-modal chat messages to the AI21 API, including both text and non-textual content. This aligns the integration with potentially new capabilities of the AI21 API, allowing for richer interactions and data handling. 📁 Total Files Changed
🧪 Test AddedN/A (No explicit tests mentioned in PR description.) 🔒Security VulnerabilitiesNo new security vulnerabilities detected. MotivationUpdated Chat Completions API endpoint, and also modified API configs and response interface. Type of Change
How Has This Been Tested?
Screenshots (if applicable)N/A Checklist
Tip Quality Recommendations
Tanka Poem ♫
Sequence DiagramsequenceDiagram
actor Client
Client->>AI21ChatCompleteConfig: Call chatComplete with Params
AI21ChatCompleteConfig->>messages.transform: Process params.messages
messages.transform-->>AI21ChatCompleteConfig: Returns transformed messages (role, content)
AI21ChatCompleteConfig->>documents.transform: Process params.messages for documents
documents.transform->>documents.transform: Extract non-text content (c.type !== 'text')
documents.transform-->>AI21ChatCompleteConfig: Returns documents array or undefined
AI21ChatCompleteConfig->>AI21 API: Send transformed messages and documents
AI21 API-->>AI21ChatCompleteConfig: ChatCompletionResponse | ErrorResponse
AI21ChatCompleteConfig-->>Client: Return response
|
src/providers/ai21/chatComplete.ts
Outdated
| max_tokens: { | ||
| param: 'maxTokens', | ||
| default: 16, | ||
| documents: { |
There was a problem hiding this comment.
you would need to handle mapping from user message content parts containing files/images or audio (whichever is supported by ai21) to the documents field
so for the fixed mapping you need to move this inside the messages object like this
messages: [{param: 'messages'...}, {param: 'documents' ...}]There was a problem hiding this comment.
in the new ai21 spec, there is no separate field for 'system', so you can remove this code block
{
param: 'system',
required: false,
transform: (params: Params) => {
if (
params.messages?.[0]?.role &&
SYSTEM_MESSAGE_ROLES.includes(params.messages?.[0]?.role)
) {
return params.messages?.[0].content;
}
},
},There was a problem hiding this comment.
also remove this chunk from the messages object
if (
params.messages?.[0]?.role &&
SYSTEM_MESSAGE_ROLES.includes(params.messages?.[0]?.role)
) {
inputMessages = params.messages.slice(1);There was a problem hiding this comment.
you will also need to map the text content part into the user and system messages from openai to ai21 format
in openai, a user can send user message like this
{
"role": "user",
"content" : [{"type": "text", "text": "abc"}, ..multiple parts]
}
but ai21 only accepts a string, so you will need to handle that
There was a problem hiding this comment.
Thanks for your review! I'm working on the requested changes and will update the pull request soon.
Also, could you please send me the link for the above-attached image for reference? As in the official API reference for AI21 Chat completions, parameters are defined similarly to the ones in my commit.
|
Important PR Review SkippedPR review skipped as per the configuration setting. Run a manually review by commenting /matter review 💡Tips to use Matter AICommand List
|
Description
Updated Chat Completions API endpoint, and also modified API configs and response interface
Motivation
This updates the existing outdated ai21 integration and fixes #1215
Type of Change
Checklist
Related Issues
Fixes #1215