The stream response with image_url is not parsed by ChatOpenAI client

[sugarforever]

Checked other resources

• This is a bug, not a usage question. For questions, please use the LangChain Forum (https://forum.langchain.com/).
• I added a very descriptive title to this issue.
• I searched the LangChain.js documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.
• I am sure that this is a bug in LangChain.js rather than my code.
• The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

See the relevant code blocks on how langchain is used.

Construct a ChatOpenAI client:

https://github.com/sugarforever/chat-ollama/blob/main/server/utils/models.ts#L59-L71

function initChat(family: string, modelName: string, params: InitChatParams, isCustomModel = false) {
  console.log(`Chat with [${family} ${modelName}]`, params.endpoint ? `, Host: ${params.endpoint}` : '')
  let endpoint = getProxyEndpoint(params.endpoint, params?.proxy || false)

  if (family === MODEL_FAMILIES.openai || isCustomModel) {
    const baseURL = openaiApiFillPath(endpoint)
    return new ChatOpenAI({
      configuration: { baseURL },
      apiKey: params.key,
      model: modelName,
    })
  }

Call stream() method.

    console.log("Streaming response")
    const transformedMessages = messages.map((message: RequestBody['messages'][number]) => {
      return [message.role, message.content]
    }) as BaseMessageLike[]
    const response = await llm?.stream(transformedMessages)

Error Message and Stack Trace (if applicable)

No response

Description

My project chat-ollama has been using LangChain.js to power the chat. I have searched and read through all the documents I can find. But looks I can't address how to receive the image_url from the OpenAI compatible API with image generation capability models, for example google/gemini-2.5-flash-image-preview:free from OpenRouter.

The following is the code for streaming response:
https://github.com/sugarforever/chat-ollama/blob/main/server/api/models/chat/index.post.ts#L444-L448

See the console log for the AI message chunk received:

ReadableStream { locked: false, state: 'readable', supportsBYOB: false }
Gathered response:  AIMessageChunk {
  "id": "gen-1756256031-2C2aiq18peW6qwnUlY2b",
  "content": "Here you go! The girl is now facing towards you.",
  "additional_kwargs": {},
  "response_metadata": {
    "prompt": 0,
    "completion": 0,
    "usage": {
      "prompt_tokens": 1702,
      "completion_tokens": 12,
      "total_tokens": 1714,
      "cost": 0,
      "is_byok": false,
      "prompt_tokens_details": {
        "cached_tokens": 0,
        "audio_tokens": 0
      },
      "cost_details": {
        "upstream_inference_cost": null,
        "input_audio_cache_read_tokens": null
      },
      "completion_tokens_details": {
        "reasoning_tokens": 0,
        "image_tokens": 0
      }
    },
    "finish_reason": "stop",
    "model_name": "google/gemini-2.5-flash-image-preview:free"
  },
  "tool_calls": [],
  "tool_call_chunks": [],
  "invalid_tool_calls": [],
  "usage_metadata": {
    "input_tokens": 1702,
    "output_tokens": 12,
    "total_tokens": 1714,
    "input_token_details": {
      "audio": 0,
      "cache_read": 0
    },
    "output_token_details": {
      "reasoning": 0
    }
  }
}

There is no image url available.

I tried with standard OpenAI node sdk with same model. I can see the image in the response.

System Info

pnpm info langchain

langchain@0.3.31 | MIT | deps: 11 | versions: 326
Typescript bindings for langchain
https://github.com/langchain-ai/langchainjs/tree/main/langchain/

keywords: llm, ai, gpt3, chain, prompt, prompt engineering, chatgpt, machine learning, ml, openai, embeddings, vectorstores

dist
.tarball: https://registry.npmjs.org/langchain/-/langchain-0.3.31.tgz
.shasum: 7ce858c98ff4921ebf4f3a59d549bcddee9c4713
.integrity: sha512-C7n7WGa44RytsuxEtGcArVcXidRqzjl6UWQxaG3NdIw4gIqErWoOlNC1qADAa04H5JAOARxuE6S99+WNXB/rzA==
.unpackedSize: 2.9 MB

dependencies:
@langchain/openai: >=0.1.0 <0.7.0 openapi-types: ^12.1.3
@langchain/textsplitters: >=0.0.0 <0.2.0 p-retry: 4
js-tiktoken: ^1.0.12 uuid: ^10.0.0
js-yaml: ^4.1.0 yaml: ^2.2.1
jsonpointer: ^5.0.1 zod: ^3.25.32
langsmith: ^0.3.46

maintainers:

• nfcampos nuno@boringbits.io
• jacoblee93 jacoblee93@gmail.com
• andrewnguonly andrewnguonly@gmail.com
• davidduong david@duong.cz
• hntrl hunter@hntrl.io
• hwchase17 hw.chase.17@gmail.com
• basproul braceasproul@gmail.com

dist-tags:
latest: 0.3.31 tag-for-publishing-older-releases: 0.2.20
next: 0.3.2-rc.0

published a week ago by hntrl hunter@hntrl.io

node --version
v23.11.0

[georgezouq]

I have the same problem, any idea to fix this?

[christian-bromann]

Fixed in #8879