Web-Bench supports custom agent capabilities by invoking your custom agent API through the built-in 'http' agent mode.

Before that, you need to complete the installation.

During the "Call Agent" step, the agent will:

1. Pass the Evaluator's context to your agent.
2. Return the your anget's response to the Evaluator without modifications.

Therefore, the request and response formats of your CustomAgent must adhere to the following interfaces:

export interface AgentRequest {
  type: 'normal' | 'init'

  task: string

  // Code files, key is filePath, value is fileContent
  files?: Record<string, string>

  // Error context
  error?: string
}


export interface AgentResponse {
  // Code files, key is filePath, value is fileContent
  files: Record<string, string>

  // [filePath:string]: string  Poor Extension
}