support video on bedrock

[narengogi]

testing done, test with base64 encoded video, make sure that documents and images are working as well

[matter-code-review]

Description

Summary By MatterAI

🔄 What Changed

This pull request introduces a new regexReplace plugin for content redaction and transformation. It significantly enhances Google and Google Vertex AI providers by adding support for inline data (e.g., video, images) in chat completions. The PR also standardizes finish_reason handling across Anthropic, Bedrock, DeepSeek, Google, Google Vertex AI, Mistral AI, and Together AI providers through new enums and updates to the transformFinishReason utility. Additionally, the DashScope API base URL has been updated, and its chatComplete parameters expanded. A minor fix corrects the OpenAI stream chunk created timestamp to be in seconds.

🔍 Impact of the Change

The new regexReplace plugin provides flexible content moderation and transformation. Multimedia support for Google and Google Vertex AI models broadens their capabilities. Standardized finish_reason improves consistency and simplifies integration for downstream services. DashScope updates ensure compatibility and expose more model parameters. The OpenAI timestamp fix ensures correct data formatting. The overall impact is a more robust, feature-rich, and standardized gateway for various LLM providers.

📁 Total Files Changed

• plugins/default/regexReplace.ts: Added a new plugin for regex-based text replacement.
• plugins/index.ts: Integrated the new regexReplace plugin into the system.
• src/providers/anthropic/chatComplete.ts: Refined tool call index initialization and increment logic for streaming.
• src/providers/anthropic/complete.ts: Updated response and stream chunk transformations for standardized finish reasons.
• src/providers/bedrock/complete.ts: Implemented standardized finish reason mapping for Bedrock Titan models.
• src/providers/bedrock/types.ts: Refactored Bedrock stop reason enums and added Titan-specific reasons.
• src/providers/dashscope/api.ts: Updated the base URL for DashScope API.
• src/providers/dashscope/index.ts: Expanded configurable parameters for DashScope chat completion.
• src/providers/deepseek/chatComplete.ts: Integrated standardized finish reason mapping for DeepSeek.
• src/providers/deepseek/types.ts: Defined new stop reason enum for DeepSeek.
• src/providers/google-vertex-ai/chatComplete.ts: Added support for inline data (multimedia) and standardized finish reasons.
• src/providers/google-vertex-ai/transformGenerationConfig.ts: Enhanced generation config to support response modalities.
• src/providers/google-vertex-ai/types.ts: Added inline data type and new finish reason enum.
• src/providers/google/chatComplete.ts: Added support for inline data (multimedia) and standardized finish reasons.
• src/providers/google/types.ts: Defined new finish reason enum for Google.
• src/providers/mistral-ai/chatComplete.ts: Integrated standardized finish reason mapping for Mistral AI.
• src/providers/mistral-ai/types.ts: Defined new finish reason enum for Mistral AI.
• src/providers/openai/chatComplete.ts: Corrected stream chunk created timestamp format.
• src/providers/together-ai/chatComplete.ts: Integrated standardized finish reason mapping for Together AI.
• src/providers/together-ai/types.ts: Defined new finish reason enum for Together AI.
• src/providers/types.ts: Centralized PROVIDER_FINISH_REASON type to include all new provider-specific enums.
• src/providers/utils/finishReasonMap.ts: Expanded finish reason mapping logic for all newly integrated providers.

🧪 Test Added

Manual testing was performed, including verification with base64 encoded video, and ensuring that existing document and image handling functionality remains intact.

🔒Security Vulnerabilities

The broader feature of supporting video on Bedrock and other providers may still have unaddressed vulnerabilities related to comprehensive input validation for file_url and file_data when handling various media types. The new regexReplace plugin, if not configured carefully, could also introduce vulnerabilities if the regex patterns are not robust or if failOnDetection is not used appropriately for sensitive data.

Motivation

To enhance the gateway's capabilities with a new regex replacement plugin, enable multimedia support for Google and Google Vertex AI, standardize finish reason handling across multiple LLM providers, update DashScope API, and fix a minor timestamp issue in OpenAI streaming.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes)

How Has This Been Tested?

• Unit Tests
• Integration Tests
• Manual Testing

Screenshots (if applicable)

N/A

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

Related Issues

N/A

Tip

Quality Recommendations

1. Implement more robust validation for the regexPattern parameter in plugins/default/regexReplace.ts to prevent potential ReDoS (Regular Expression Denial of Service) attacks, especially given the g flag. Consider adding a timeout or complexity limit for regex execution.

2. Strengthen input validation for inlineData (e.g., base64 encoded video/images) across Google and Google Vertex AI providers to ensure proper format, size limits, and content type checks, mitigating potential abuse or malformed data issues.

3. Add dedicated unit tests for src/providers/utils/finishReasonMap.ts to ensure all new and existing provider-specific finish reasons are correctly mapped to the standardized FINISH_REASON enum, covering all edge cases and future additions.

4. Consider more specific error types or logging for different failure scenarios within the regexReplace plugin (e.g., invalid regex syntax vs. missing content) for better debugging and operational visibility.

Tanka Poem ♫

New patterns emerge,
Video flows, reasons clear,
Data's path defined.
Gateway transforms, ever keen,
Knowledge shared, a future bright. ✨

Sequence Diagram

sequenceDiagram
    participant Client as Client
    participant Gateway as Gateway
    participant RegexPlugin as Regex Replace Plugin
    participant Utils as Gateway Utils
    participant LLMProvider as LLM Provider (e.g., Google, Anthropic)

    Client->>+Gateway: API Request (params, content, modalities)
    Note over Gateway: Process Request

    opt Regex Plugin Configured
        Gateway->>+RegexPlugin: handle(context, params, eventType)
        RegexPlugin->>Utils: getCurrentContentPart(context, eventType)
        Utils-->>RegexPlugin: {content, textArray}
        Note over RegexPlugin: Apply regex, redact/transform text
        alt Matches Found
            RegexPlugin->>Utils: setCurrentContentPart(context, eventType, transformedData, mappedTextArray)
            Utils-->>RegexPlugin: Content Updated
        end
        RegexPlugin-->>-Gateway: {verdict, data, transformed}
    end

    Gateway->>+LLMProvider: Forward Request (transformed content)
    LLMProvider-->>-Gateway: Raw LLM Response (stream/complete)

    alt Streaming Response
        loop Each Stream Chunk
            Gateway->>+LLMProvider: Stream Chunk
            Note over Gateway: Transform Stream Chunk
            Gateway->>Utils: transformFinishReason(providerReason, strictOpenAiCompliance)
            Utils-->>Gateway: Standardized Finish Reason
            Gateway-->>Client: Transformed Stream Chunk
        end
    else Complete Response
        Note over Gateway: Transform Complete Response
        Gateway->>Utils: transformFinishReason(providerReason, strictOpenAiCompliance)
        Utils-->>Gateway: Standardized Finish Reason
        Gateway-->>Client: Transformed Complete Response
    end

Loading

[matter-code-review]

Video support implementation looks solid with proper type definitions and MIME type handling. Minor improvements needed for error handling and type safety.

[matter-code-review]

Important

PR Review Skipped

PR review skipped as per the configuration setting. Run a manually review by commenting /matter review

💡Tips to use Matter AI

Command List

• /matter summary: Generate AI Summary for the PR
• /matter review: Generate AI Reviews for the latest commit in the PR
• /matter review-full: Generate AI Reviews for the complete PR
• /matter release-notes: Generate AI release-notes for the PR
• /matter : Chat with your PR with Matter AI Agent
• /matter remember : Generate AI memories for the PR
• /matter explain: Get an explanation of the PR
• /matter help: Show the list of available commands and documentation
• Need help? Join our Discord server: https://discord.gg/fJU5DvanU3

[matter-code-review]

Important

PR Review Skipped

PR review skipped as per the configuration setting. Run a manually review by commenting /matter review

💡Tips to use Matter AI

Command List

• /matter summary: Generate AI Summary for the PR
• /matter review: Generate AI Reviews for the latest commit in the PR
• /matter review-full: Generate AI Reviews for the complete PR
• /matter release-notes: Generate AI release-notes for the PR
• /matter : Chat with your PR with Matter AI Agent
• /matter remember : Generate AI memories for the PR
• /matter explain: Get an explanation of the PR
• /matter help: Show the list of available commands and documentation
• Need help? Join our Discord server: https://discord.gg/fJU5DvanU3

[matter-code-review]

Comprehensive video support implementation with proper type safety and finish reason standardization across providers