Allow getting and setting content part for /messages routes

[roh26it]

Added snitch tests for it here - https://github.com/Portkey-AI/snitch-tests/commit/cf9944d97292afc8341f2ec10929a2c38983a537

[matter-code-review]

Description

Summary By MatterAI

🔄 What Changed

This pull request primarily introduces support for the new 302ai provider, including its API configuration, chat completion parameters, and response/stream transformation logic. A significant refactoring effort was undertaken to standardize how providerOptions are passed throughout the request handling and transformation pipeline, moving from simple string provider parameters to a more comprehensive Options object. This involved updating responseHandler, ResponseService, and various transformUsingProviderConfig calls across multiple provider modules (Bedrock, Google Vertex AI, OpenAI, Azure OpenAI). Additionally, several new parameters (background, prompt, stream_options, prompt_cache_key, safety_identifier, verbosity) have been added to existing OpenAI and Azure OpenAI chat completion configurations, enhancing their flexibility.

🔍 Impact of the Change

The integration of the 302ai provider expands the gateway's capabilities, allowing users to leverage another AI model. The refactoring to consistently use providerOptions improves the maintainability, extensibility, and clarity of the codebase, making it easier to manage provider-specific configurations and parameters. This change ensures that all relevant provider-specific information is encapsulated and passed uniformly, reducing potential for errors and simplifying future additions or modifications to provider integrations. The new parameters for existing providers offer more granular control over model behavior.

📁 Total Files Changed

• package.json: Version bumped to 1.11.1.
• src/globals.ts: Added THREE_ZERO_TWO_AI to valid providers.
• src/handlers/responseHandlers.ts: Refactored responseHandler to accept providerOptions object.
• src/handlers/services/responseService.ts: Updated responseHandler call to pass this.context.providerOption.
• src/providers/302ai/api.ts: Added API configuration for 302ai provider.
• src/providers/302ai/chatComplete.ts: Implemented chat completion configuration and response/stream transforms for 302ai.
• src/providers/302ai/index.ts: Registered 302ai provider configurations.
• src/providers/azure-openai/chatComplete.ts: Added new parameters like prompt_cache_key, safety_identifier, verbosity.
• src/providers/bedrock/api.ts: Updated getConfig to use providerOptions.foundationModel.
• src/providers/bedrock/index.ts: Refactored getConfig to accept params and providerOptions.
• src/providers/bedrock/uploadFile.ts: Updated transformUsingProviderConfig to pass providerOptions.
• src/providers/google-vertex-ai/index.ts: Refactored getConfig to accept params.
• src/providers/google-vertex-ai/uploadFile.ts: Updated transformUsingProviderConfig to pass providerOptions.
• src/providers/index.ts: Registered the new 302ai provider.
• src/providers/open-ai-base/createModelResponse.ts: Added new parameters like background, prompt, stream_options, prompt_cache_key, verbosity.
• src/providers/openai/chatComplete.ts: Added new parameters like prompt_cache_key, safety_identifier, verbosity.
• src/providers/stability-ai/index.ts: Refactored getConfig to accept params.
• src/providers/types.ts: Updated ProviderConfigs interface to include getConfig signature and Options interface to make provider non-optional and add foundationModel.
• src/services/transformToProviderRequest.ts: Major refactoring to consistently pass providerOptions to transformation functions.
• src/types/requestBody.ts: Made provider non-optional in Options and added foundationModel.

🧪 Test Added

N/A

🔒Security Vulnerabilities

N/A

Motivation

This change is motivated by the need to expand the gateway's capabilities by integrating a new AI provider (302ai) and to improve the internal architecture's consistency and maintainability by standardizing the passing of provider-specific configuration options throughout the request processing pipeline.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes)

How Has This Been Tested?

• Unit Tests
• Integration Tests
• Manual Testing

Screenshots (if applicable)

N/A

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

Related Issues

N/A

Tip

Quality Recommendations

1. Implement comprehensive unit and integration tests for the newly integrated 302ai provider to ensure its functionality, error handling, and streaming capabilities work as expected.

2. Address the TODO comment in src/services/transformToProviderRequest.ts to fully refactor the file to consistently use the providerOptions object, improving code clarity and maintainability.

3. Ensure that newly added parameters like prompt_cache_key, safety_identifier, verbosity, background, prompt, and stream_options in OpenAICreateModelResponseConfig, OpenAIChatCompleteConfig, and AzureOpenAIChatCompleteConfig have appropriate input validation at the gateway level if they are user-controlled, to prevent unexpected behavior or potential security issues.

Tanka Poem ♫

New AI awakes,
Options flow, a cleaner stream,
Code refactored, bright.
Data moves with purpose clear,
Gateway hums, a future built. ✨

[matter-code-review]

Important

PR Review Skipped

PR review skipped as per the configuration setting. Run a manually review by commenting /matter review

💡Tips to use Matter AI

Command List

• /matter summary: Generate AI Summary for the PR
• /matter review: Generate AI Reviews for the latest commit in the PR
• /matter review-full: Generate AI Reviews for the complete PR
• /matter release-notes: Generate AI release-notes for the PR
• /matter : Chat with your PR with Matter AI Agent
• /matter remember : Generate AI memories for the PR
• /matter explain: Get an explanation of the PR
• /matter help: Show the list of available commands and documentation
• Need help? Join our Discord server: https://discord.gg/fJU5DvanU3