mistral on vertex #1314
mistral on vertex #1314
Conversation
|
DescriptionSummary By MatterAI
🔄 What ChangedThis pull request introduces two new default plugins: 🔍 Impact of the ChangeThe changes enhance the gateway's capabilities for content moderation, model governance, and multimedia processing, making it more versatile for various AI applications. The addition of Meshy and Tripo3D expands the platform's offering to 3D generation. Improved streaming error handling increases system reliability, and the extended conditional routing provides greater control over request flow. The previous PR's cleanup of an unused 📁 Total Files Changed
🧪 Test AddedN/A (No explicit tests are mentioned in the PR JSON. It is recommended to add comprehensive unit and integration tests for the new plugins and provider integrations to ensure functionality and prevent regressions.) 🔒Security VulnerabilitiesN/A (No explicit security vulnerabilities are introduced or addressed by this change, but proper input validation for regex patterns is crucial to prevent ReDoS attacks in the MotivationTo enhance the gateway's functionality with new content moderation and model governance plugins, expand multimedia support for existing AI providers, and integrate new 3D generation services, thereby broadening the platform's capabilities and improving overall system robustness. Type of Change
How Has This Been Tested?
Screenshots (if applicable)N/A Checklist
Related IssuesN/A Tip Quality Recommendations
Tanka Poem ♫
Sequence DiagramsequenceDiagram
participant User as User
participant Gateway as API Gateway
participant PluginHandler as Plugin Handler
participant ConditionalRouter as Conditional Router
participant BedrockAPI as AWS Bedrock API
participant GoogleVertexAIAPI as Google Vertex AI API
participant MeshyAPI as Meshy API
participant Tripo3DAPI as Tripo3D API
User->>+Gateway: API Request (e.g., /chat/completions, /text-to-3d)
Note over Gateway: Process incoming request
alt Request for Model Rules or Regex Replacement
Gateway->>+PluginHandler: Invoke Plugin (context, params, eventType)
Note over PluginHandler: plugins/default/modelRules.ts
PluginHandler->>PluginHandler: Evaluate model access rules (requestModel, metadata, rulesConfig)
PluginHandler-->>-Gateway: {error, verdict, data}
Gateway->>+PluginHandler: Invoke Plugin (context, params, eventType)
Note over PluginHandler: plugins/default/regexReplace.ts
PluginHandler->>PluginHandler: Apply regex replacement (content, regexPattern, redactText)
PluginHandler-->>-Gateway: {error, verdict, data, transformedData, transformed}
end
Gateway->>+ConditionalRouter: Resolve Target (currentTarget, metadata, params, url.pathname)
ConditionalRouter-->>-Gateway: finalTarget
alt Route to Bedrock
Gateway->>+BedrockAPI: Chat Completion Request (messages with text, image, video)
Note over BedrockAPI: src/providers/bedrock/chatComplete.ts
BedrockAPI-->>-Gateway: Chat Completion Response
else Route to Google/Vertex AI
Gateway->>+GoogleVertexAIAPI: Chat Completion Request (messages with text, inlineData, modalities)
Note over GoogleVertexAIAPI: src/providers/google-vertex-ai/chatComplete.ts
GoogleVertexAIAPI-->>-Gateway: Chat Completion Response (with inlineData)
else Route to Meshy
Gateway->>+MeshyAPI: 3D Generation Request (providerOptions.apiKey)
Note over MeshyAPI: src/providers/meshy/api.ts
MeshyAPI-->>-Gateway: 3D Generation Response
else Route to Tripo3D
Gateway->>+Tripo3DAPI: 3D Generation Request (providerOptions.apiKey)
Note over Tripo3DAPI: src/providers/tripo3d/api.ts
Tripo3DAPI-->>-Gateway: 3D Generation Response
end
alt Streaming Response
Gateway->>+User: Stream Data Chunks
loop For each chunk
Gateway->>Gateway: Encode chunk
Gateway->>User: Write encoded chunk
end
alt Error during stream
Gateway->>Gateway: Log 'Error during stream processing'
else Finally
Gateway->>Gateway: Attempt to close writer
alt Close writer error
Gateway->>Gateway: Log 'Failed to close the writer'
end
end
else Non-Streaming Response
Gateway-->>-User: Final API Response
end
Gateway-->>-User: Final Response
|
![matter-code-review[bot]](https://avatars.githubusercontent.com/in/474549?s=60&v=4){kind=small}
|
Clean import removal - no issues found in this minimal change. |
|
Code quality is good overall with proper TypeScript patterns. Found several critical bugs including null reference crashes, type safety issues, and resource management problems that need immediate attention. |
2 participants
sample request body
{ "model": "mistralai.codestral-2501", "max_tokens": 1000, "stream": false, "messages": [ { "role": "system", "content": [ { "type": "text", "text": "You are a helpful assistant" } ] }, { "role": "user", "content": "I'm in Chennai, what is the time and what is the temperature?" } ], "tools": [ { "type": "function", "function": { "name": "get_current_temperature_in_sf", "description": "Get the current temperature for a specific location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g., San Francisco, CA" }, "temperature": { "type": "number", "exclusiveMinimum": 0, "description": "The minimum temperature in degrees, must be strictly greater than 0" } }, "required": [ "location", "temperature" ] } } }, { "type": "function", "function": { "name": "get_current_time", "description": "Get the current time forr a specific location", "parameters": { "type": "object", "properties": { "location": { "type": "string", "description": "The city and state, e.g., San Francisco, CA" } }, "required": [ "location" ] } } } ] }