Skip to content

Conversation

christian-bromann
Copy link
Member

Fix tool call streaming for empty schemas

Fixes a bug where tool calls with empty schemas (z.object({})) were being dropped in streaming mode but worked correctly in non-streaming mode.

Problem

The issue occurred because AIMessageChunk constructor was filtering out tool call chunks that didn't have an ID. OpenAI sometimes sends chunks without IDs for tools with empty parameter schemas, causing these tool calls to be completely lost in streaming mode while working fine in non-streaming mode.

Solution

Changes:

  • Modified tool call chunk grouping logic in langchain-core/src/messages/ai.ts to assign fallback IDs for chunks without IDs (format: "fallback-{index}")
  • Preserve original IDs when present to maintain backward compatibility
  • Added comprehensive unit tests for chunks without IDs in langchain-core/src/messages/tests/base_message.test.ts
  • Added integration test in libs/langchain-openai/src/tests/chat_models-extended.int.test.ts to verify streaming/non-streaming parity

Result

Tool calls now work consistently in both streaming and non-streaming modes for tools with empty schemas. Users can reliably use parameter-less tools like get_current_time in streaming contexts.

Before:

// Streaming mode - tool calls lost ❌
const stream = await llmWithTools.stream(messages);
// finalChunk.tool_calls === []

After:

// Streaming mode - tool calls preserved ✅  
const stream = await llmWithTools.stream(messages);
// finalChunk.tool_calls === [{ name: "get_current_time", args: {}, id: "fallback-0" }]

Fixes #8518

Fixes a bug where tool calls with empty schemas (z.object({})) were
being dropped in streaming mode but worked correctly in non-streaming mode.

The issue occurred because AIMessageChunk constructor was filtering out
tool call chunks that didn't have an ID. OpenAI sometimes sends chunks
without IDs for tools with empty parameter schemas.

Changes:
- Modified tool call chunk grouping logic to assign fallback IDs
  for chunks without IDs (format: "fallback-{index}")
- Preserve original IDs when present
- Added unit tests for chunks without IDs
- Added integration test to verify streaming/non-streaming parity

Tool calls now work consistently in both streaming and
non-streaming modes for tools with empty schemas.

fixes langchain-ai#8518
Copy link

vercel bot commented Jul 29, 2025

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchainjs-docs ✅ Ready (Inspect) Visit Preview Jul 29, 2025 1:22am
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
langchainjs-api-refs ⬜️ Ignored (Inspect) Jul 29, 2025 1:22am

@hntrl hntrl merged commit 9212f72 into langchain-ai:main Jul 30, 2025
38 checks passed
@christian-bromann christian-bromann deleted the cb/tool-call-id branch September 17, 2025 16:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

When LangChain makes a streaming call to a function with no parameters, it does not receive a toolcall response.

2 participants