unified route handler for image edits

[narengogi]

example payload

curl --location 'http://localhost:8787/v1/images/edits' \
--header 'Content-Type: multipart/form-data' \
--header 'Authorization: Bearer sk-' \
--header 'x-portkey-provider: openai' \
--header 'x-portkey-api-key: ' \
--form 'model="dall-e-2"' \
--form 'image=@"/Users/naren/Desktop/Screenshot 2025-09-12 at 4.08.29 PM.png"' \
--form 'prompt="add a comic character to the image"'

[matter-code-review]

Summary By MatterAI

🔄 What Changed

This Pull Request introduces a new allowedRequestTypes plugin for granular control over API request types, a dedicated messagesCountTokensHandler for token counting, and integrates NextBit as a new AI provider. It also enhances the regexReplace plugin with a safer regex parsing utility, improves WebSocket error handling, and adds azureEntraScope support for Azure AI Inference. Various provider configurations (Anthropic, Bedrock, Google Vertex AI, OpenAI) have been updated to support the new token counting functionality and other specific parameters like performance_config and stream_options.

🔍 Impact of the Change

These changes significantly expand the gateway's capabilities by offering fine-grained control over allowed/blocked request types, providing a standardized way to count tokens across different AI providers, and broadening the range of supported AI services. The introduction of safe regex parsing improves security and robustness, while WebSocket enhancements contribute to better connection stability. The updates ensure the gateway remains flexible and extensible for future AI integrations and operational requirements.

📁 Total Files Changed

• package.json: Updated @hono/node-ws dependency to ^1.2.0.
• plugins/default/allowedRequestTypes.ts: Added a new plugin to filter requests based on allowed/blocked types from parameters or metadata.
• plugins/default/default.test.ts: Added comprehensive unit tests for the new allowedRequestTypes plugin, covering various scenarios including allowlist, blocklist, combined modes, and error handling. Also, a minor import path fix for modelWhitelist.
• plugins/default/regexReplace.ts: Introduced a parseRegex utility function to safely parse regex patterns and validate flags, improving the robustness of the regexReplace handler.
• plugins/index.ts: Registered the new allowedRequestTypes plugin for use within the gateway.
• src/globals.ts: Added NEXTBIT as a new valid AI provider.
• src/handlers/handlerUtils.ts: Extended constructConfigFromRequestHeaders to include azureEntraScope.
• src/handlers/messagesCountTokensHandler.ts: Added a new handler to process /messages/count_tokens requests, leveraging tryTargetsRecursively.
• src/handlers/realtimeHandlerNode.ts: Implemented a mechanism to wait for the upstream WebSocket to be open before proceeding, improving connection reliability.
• src/handlers/services/responseService.ts: Refactored content-encoding header removal logic specifically for the Node.js runtime.
• src/handlers/websocketUtils.ts: Enhanced WebSocket error handling by adding a dedicated errorListener and ensuring its proper removal.
• src/index.ts: Registered the new /v1/messages/count_tokens POST route with the messagesCountTokensHandler.
• src/providers/anthropic/api.ts: Added endpoint mapping for messagesCountTokens.
• src/providers/anthropic/index.ts: Included messagesCountTokens in Anthropic provider configuration.
• src/providers/azure-ai-inference/api.ts: Integrated azureEntraScope into the Azure Entra ID token acquisition process.
• src/providers/bedrock/api.ts: Added endpoint mapping for messagesCountTokens.
• src/providers/bedrock/chatComplete.ts: Added performance_config parameter to Bedrock chat completion configuration.
• src/providers/bedrock/countTokens.ts: Added a new file defining the configuration and response transformation for Bedrock's messagesCountTokens functionality.
• src/providers/bedrock/index.ts: Integrated messagesCountTokens configuration and response transform, updated model replacement regex to include apac., and refactored default configuration application.
• src/providers/bedrock/messages.ts: Added performance_config parameter to Bedrock messages configuration.
• src/providers/bedrock/uploadFile.ts: Improved model parameter handling to support ARN and added titan provider to getProviderConfig.
• src/providers/google-vertex-ai/api.ts: Added endpoint mapping for messagesCountTokens.
• src/providers/google-vertex-ai/chatComplete.ts: Modified response validation to also consider usageMetadata for valid responses.
• src/providers/google-vertex-ai/index.ts: Integrated messagesCountTokens into Google Vertex AI provider configuration.
• src/providers/google-vertex-ai/messagesCountTokens.ts: Added a new file defining the configuration for Google Vertex AI's messagesCountTokens.
• src/providers/index.ts: Registered the new NextBitConfig.
• src/providers/nextbit/api.ts: Added a new file defining the API configuration for the NextBit provider.
• src/providers/nextbit/index.ts: Added a new file defining the overall configuration for the NextBit provider.
• src/providers/openai/complete.ts: Added stream_options parameter to OpenAI complete configuration.
• src/providers/types.ts: Extended the API type to include messagesCountTokens operation.
• src/services/transformToProviderRequest.ts: Made providerOptions optional in transformUsingProviderConfig.
• src/types/requestBody.ts: Added azureEntraScope to the Options interface.

🧪 Test Added

Extensive unit tests were added for the new allowedRequestTypes plugin in plugins/default/default.test.ts. These tests cover:

• Allowing requests when the type is in the allowedTypes list (array and comma-separated string).
• Rejecting requests when the type is not in the allowedTypes list.
• Handling streaming request types correctly.
• Prioritizing parameters over metadata for configuration.
• Default behavior of allowing all request types when no restrictions are configured.
• Error handling for missing requestType in context.
• Blocking request types specified in blockedTypes (array and comma-separated string).
• Allowing request types not in the blockedTypes list.
• Combining allowedTypes and blockedTypes with correct precedence.
• Detecting and erroring on conflicts where a type appears in both allowedTypes and blockedTypes.

🔒Security Vulnerabilities

N/A (The parseRegex function addition is a security improvement, not a vulnerability fix).

Motivation

To enhance gateway functionality with request type filtering, token counting, and new provider support, while improving robustness and stability.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes)

How Has This Been Tested?

• Unit Tests
• Integration Tests
• Manual Testing

Screenshots (if applicable)

N/A

Checklist

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have added tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

Related Issues

.

Tip

Quality Recommendations

1. Consider adding integration tests for the new messagesCountTokensHandler across different providers to ensure end-to-end functionality and compatibility.

2. Ensure comprehensive documentation for the new allowedRequestTypes plugin, including clear examples for both array and comma-separated string inputs for allowedTypes and blockedTypes parameters.

3. The parseRegex utility function could be extracted to a shared utility file if other plugins or handlers might benefit from safe regex parsing in the future, promoting code reuse.

Tanka Poem ♫

New routes now flow,
Tokens counted, types allowed,
NextBit joins the fray.
WebSockets stand firm and true,
Gateway grows, robust and new. ✨

Sequence Diagram

sequenceDiagram
    participant Client
    participant Gateway as Portkey Gateway
    participant Handler as messagesCountTokensHandler
    participant Utils as handlerUtils
    participant ProviderAPI as AI Provider API
    
    Client->>Gateway: POST /v1/messages/count_tokens (requestBody, headers)
    Gateway->>Handler: messagesCountTokensHandler(c: Context)
    Handler->>Utils: constructConfigFromRequestHeaders(requestHeaders)
    Utils-->>Handler: camelCaseConfig
    Handler->>Utils: tryTargetsRecursively(c, camelCaseConfig, request, requestHeaders, 'messagesCountTokens', 'POST', 'config')
    Utils->>ProviderAPI: POST /model/{model}/count-tokens (transformed request)
    ProviderAPI-->>Utils: 200 OK (token count response)
    Utils-->>Handler: tryTargetsResponse
    Handler-->>Gateway: Response
    Gateway-->>Client: 200 OK (token count)
    
    alt Error during processing
        Handler-->>Handler: Log error
        Handler-->>Gateway: 400/500 Error Response
        Gateway-->>Client: Error Response
    end

Loading

[matter-code-review]

Added image edits handler with unified route processing. Found one critical bug in error response.

[matter-code-review]

PR introduces a new allowedRequestTypes plugin with comprehensive tests, adds messagesCountTokens support for multiple providers, and includes various bug fixes and enhancements.