Skip to content

fix: enable handleLLMNewToken events for useResponsesApi=true #8578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 3 commits into from
Aug 4, 2025
Merged

fix: enable handleLLMNewToken events for useResponsesApi=true #8578

merged 3 commits into from
Aug 4, 2025

Conversation

ninjudd
Copy link
Contributor

@ninjudd ninjudd commented Jul 28, 2025

  • Fix ChatOpenAI._streamResponseChunks to pass runManager to responses._streamResponseChunks
  • Fix ChatOpenAIResponses._streamResponseChunks to accept and use runManager parameter
  • Add handleLLMEnd event emission for non-streaming responses
  • Add regression test to verify handleLLMNewToken events are emitted
  • Resolves token-by-token streaming issue when useResponsesApi=true is enabled

Fixes #8577

@ninjudd
fix: enable handleLLMNewToken events for useResponsesApi=true
fa9f31a 
- Fix ChatOpenAI._streamResponseChunks to pass runManager to responses._streamResponseChunks
- Fix ChatOpenAIResponses._streamResponseChunks to accept and use runManager parameter
- Add handleLLMEnd event emission for non-streaming responses
- Add regression test to verify handleLLMNewToken events are emitted
- Resolves token-by-token streaming issue when useResponsesApi=true is enabled
@vercel Vercel
Copy link

vercel bot commented Jul 28, 2025 •
edited
Loading

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchainjs-docs ✅ Ready (Inspect) Visit Preview Aug 4, 2025 7:48am
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
langchainjs-api-refs ⬜️ Ignored (Inspect) Aug 4, 2025 7:48am

@ninjudd ninjudd mentioned this pull request Jul 28, 2025
Closed
5 tasks
@ninjudd
Copy link
Contributor Author

ninjudd commented Jul 28, 2025

I believe this PR also fixes #8283

hntrl
hntrl reviewed Jul 31, 2025
View reviewed changes
Copy link
Member

@hntrl hntrl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice catch @ninjudd! I don't believe this is something that was ever supported. The writeup + PR is very much appreciated ❤️

One note I had about handleLLMEnd, and I don't think the params being passed to handleLLMNewToken are exactly accurate. I'll pick up on this tomorrow since I think it's a little tricker then what is initially let on.

libs/langchain-openai/src/chat_models.ts Outdated
Comment on lines 1543 to 1550

// Emit handleLLMEnd event for non-streaming responses
await runManager?.handleLLMEnd({
generations: [result.generations],
llmOutput: result.llmOutput,
});

return result;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't believe this is necessary? This gets handled upstream from the base chat model (if it doesn't it should, I'll take a look)

https://github.com/ninjudd/langchainjs/blob/fa9f31a90212a7ed41f998e89a59099e971b3981/langchain-core/src/language_models/chat_models.ts#L479-L482

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, you may be right! Given that everything but streaming works without this PR, emitting handleLLMEnd here is probably not necessary. I can update the PR to remove this if you agree.

@hntrl hntrl linked an issue Aug 4, 2025 that may be closed by this pull request
Unable to stream output when using OpenAI's Responses API to call the web_search tool #8283
Closed
5 tasks
@hntrl hntrl merged commit cc4ff80 into langchain-ai:main Aug 4, 2025
31 of 32 checks passed
@ninjudd
Copy link
Contributor Author

ninjudd commented Aug 4, 2025

Thanks @hntrl! Any idea when the next release will go out?

Also, where do I track that? Looks like the last tagged release on Github was June 19.

@hntrl
Copy link
Member

hntrl commented Aug 7, 2025

Hey @ninjudd! Apologies for the delay! We haven't dont a good job of posting our releases as of late. This was published under @langchain/openai==0.6.4

hntrl added a commit that referenced this pull request Oct 14, 2025
@hntrl
chore(openai/v1): patch changes
990a78d 
* #9159
* #9157
* #9151
* #9145
* #9069
* #9043
* #9019
* #9002
* #8995
* #8974
* #8916
* #8880
* #8874
* #8876
* #8839
* #8837
* #8807
* #8754
* #8711
* #8715
* #8656
* #8654
* #8651
* #8650
* #8649
* #8645
* #8646
* #8647
* #8640
* #8639
* #8633
* #8578
@hntrl hntrl mentioned this pull request Oct 14, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hntrl hntrl hntrl left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Streaming Does not Work with OpenAI Responses API Unable to stream output when using OpenAI's Responses API to call the web_search tool

2 participants

@ninjudd @hntrl