vMCP: Implement backend client pooling to reuse MCP connections within sessions

[JAORMX]

Problem

vMCP creates a fresh MCP client for every backend operation:

// pkg/vmcp/client/client.go:302-320
func (h *httpBackendClient) CallTool(...) {
    c, err := h.clientFactory(ctx, target)  // New client every time
    defer c.Close()
    
    initializeClient(ctx, c)                 // Repeat handshake
    result, err := c.CallTool(...)          // Single operation
}

Impact:

• Repeats MCP initialization handshake on every operation
• Violates MCP spec session lifecycle best practices
• 50-80% unnecessary overhead (new connection + capability negotiation)
• Loses backend server-side session state

Findings

1. We Already Have Session Management

• ✅ pkg/transport/session/Manager tracks vMCP sessions with TTL cleanup
• ✅ pkg/vmcp/server/session_adapter.go integrates with mark3labs SDK
• ✅ Sessions work correctly for client-facing connections

2. mark3labs SDK Provides Session Context

The SDK injects session info into every handler's context:

import "github.com/mark3labs/mcp-go/server"

func handler(ctx context.Context, request mcp.CallToolRequest) {
    session := server.ClientSessionFromContext(ctx)
    sessionID := session.SessionID()  // Available in every handler!
}

3. Backend Clients Are Always Local

MCP clients contain TCP connections and goroutines - they cannot be serialized. They must live in-memory even if session metadata is in Redis.

Proposed Solution

Create a BackendClientPool that:

1. Maps sessionID → (backendID → *client.Client)
2. Reuses initialized clients within a vMCP session
3. Cleans up when vMCP session expires/terminates
4. Lives in-memory on each vMCP instance

Architecture

vMCP Session (tracked by session.Manager)
    │
    ├─ Backend Client Pool (NEW)
    │   ├─ Backend A Client (reused)
    │   ├─ Backend B Client (reused)
    │   └─ Backend C Client (reused)
    │
    └─ Cleanup on session termination

Implementation Sketch

// pkg/vmcp/client/pool.go
type BackendClientPool struct {
    clients map[string]map[string]*client.Client  // sessionID → backendID → Client
    mu      sync.RWMutex
}

func (p *BackendClientPool) GetOrCreateClient(
    ctx context.Context,
    sessionID string,
    target *vmcp.BackendTarget,
) (*client.Client, error) {
    // Check pool first
    if client := p.getFromPool(sessionID, target.WorkloadID); client != nil {
        return client, nil  // Reuse!
    }
    
    // Create, initialize, and cache
    client := p.factory(ctx, target)
    initializeClient(ctx, client)
    p.storeInPool(sessionID, target.WorkloadID, client)
    return client, nil
}

func (p *BackendClientPool) CleanupSession(ctx context.Context, sessionID string) error {
    // Close all backend clients for this session
}

Integration Points

1. Extract session ID in handlers:

// pkg/vmcp/server/server.go
func (s *Server) createToolHandler(toolName string) func(...) {
    return func(ctx context.Context, request mcp.CallToolRequest) {
        sessionID := extractSessionID(ctx)  // From SDK context
        target, _ := s.router.RouteTool(ctx, toolName)
        
        // Use pooled client
        client, _ := s.backendClientPool.GetOrCreateClient(ctx, sessionID, target)
        result, _ := client.CallTool(...)  // No initialization needed!
    }
}

2. Cleanup on session termination:

// pkg/vmcp/server/session_adapter.go
func (a *sessionIDAdapter) Terminate(sessionID string) (bool, error) {
    // ... existing logic ...
    
    // Cleanup backend clients
    a.backendClientPool.CleanupSession(context.Background(), sessionID)
    return false, nil
}

Benefits

• ✅ 50-80% fewer backend initialization calls
• ✅ 20-40% latency reduction for operations within a session
• ✅ MCP spec compliant session lifecycle
• ✅ Preserves backend state across operations
• ✅ Simple implementation - reuses existing session infrastructure

Files to Create/Modify

New:

• pkg/vmcp/client/pool.go
• pkg/vmcp/client/pool_test.go

Modified:

• pkg/vmcp/server/server.go - Wire pool, extract session ID
• pkg/vmcp/server/session_adapter.go - Add cleanup hook
• pkg/vmcp/client/client.go - Use pool instead of one-shot clients

Future: Multi-Instance with Redis

When Redis session storage is added, the pool stays in-memory (connections can't be serialized):

Redis: Session metadata only {id, timestamps}
         │
    ┌────┼────┐
    ▼    ▼    ▼
 vMCP-1 vMCP-2 vMCP-3
    │    │    │
    └─ BackendClientPool (local on each instance)

If a request moves to a different instance, clients are recreated (acceptable one-time cost).

Open Questions

1. Does session.Manager support expiration callbacks? (May need to add for pool cleanup)
2. Should ListCapabilities use pooling? (Some backends require auth for discovery)
3. Fallback strategy if sessionID is empty? (Create one-shot client for backwards compatibility)

---

Effort: 2-3 days
Priority: High (performance + MCP compliance)