feat(langchain/createAgent): llmToolSelectorMiddleware implementation #9050
Conversation
|
|
The latest updates on your projects. Learn more about Vercel for GitHub.
1 Skipped Deployment
|
christian-bromann
force-pushed
the
cb/tools-in-modelrequest
branch
from
d2b8115 to
ab79c2b
Compare
christian-bromann
force-pushed
the
cb/big-tool-new-for-real
branch
from
58d2851 to
83ed806
Compare
christian-bromann
force-pushed
the
cb/tools-in-modelrequest
branch
from
ab79c2b to
4350887
Compare
christian-bromann
force-pushed
the
cb/big-tool-new-for-real
branch
from
83ed806 to
c40627d
Compare
christian-bromann
force-pushed
the
cb/big-tool-new-for-real
branch
from
c40627d to
0eed1f9
Compare
| * }); | ||
| * ``` | ||
| */ | ||
| export function llmToolSelectorMiddleware(options: LLMToolSelectorConfig) { |
There was a problem hiding this comment.
why not bigtool? Just curious
There was a problem hiding this comment.
I got a Python implementation from @eyurtsev and copied the name from there, (changed file name later on)
There was a problem hiding this comment.
I'd prefer renaming to toolSelection.ts personally and put some different strategies here in the long run... unless we think that bigTool.ts is obvious
| const toolRepresentation = toolInfo | ||
| .map(({ name, description }) => `- ${name}: ${description}`) | ||
| .join("\n"); |
There was a problem hiding this comment.
Should this be configurable?
There was a problem hiding this comment.
I guess this goes into prompt engineering - maybe there are ways you would want to represent tools in different ways, e.g. I could imagine some models like an XML description of a tool better?
| const model = runtime.context.model ?? options.model; | ||
| const maxTools = runtime.context.maxTools ?? options.maxTools; | ||
| const includeFullHistory = | ||
| runtime.context.includeFullHistory === DEFAULT_INCLUDE_FULL_HISTORY | ||
| ? options.includeFullHistory ?? runtime.context.includeFullHistory | ||
| : runtime.context.includeFullHistory ?? options.includeFullHistory; | ||
| const maxRetries = | ||
| runtime.context.maxRetries === DEFAULT_MAX_RETRIES | ||
| ? options.maxRetries ?? runtime.context.maxRetries | ||
| : runtime.context.maxRetries ?? options.maxRetries; | ||
| const defaultSystemPrompt = | ||
| runtime.context.systemPrompt === DEFAULT_SYSTEM_PROMPT | ||
| ? options.systemPrompt ?? runtime.context.systemPrompt | ||
| : runtime.context.systemPrompt ?? options.systemPrompt; |
There was a problem hiding this comment.
In the short term could these just be args to middleware? not fallback to runtime context spec?
gives users 1 way to do things for now
| if (includeFullHistory) { | ||
| const userMessages = request.messages | ||
| .filter(HumanMessage.isInstance) | ||
| .map((msg: BaseMessage) => msg.content) |
There was a problem hiding this comment.
why are we only keeping HumanMessages for the full history? the full history is usually meant to include also AImessage and ToolMessage
| /** | ||
| * Get the latest user message | ||
| */ | ||
| const latestMessage = request.messages.at(-1); |
There was a problem hiding this comment.
I believe that there's a bug here -- the latest message may not be a user message (it may be a ToolMessage).
| ); | ||
|
|
||
| if (selectedToolNames.length === 0) { | ||
| systemMessage += `\n\nNote: You have not selected any tools. Please select at least one tool.`; |
There was a problem hiding this comment.
This does not appear to be correct behavior. I think it's OK to select 0 tools if there are no relevant tools.
| export function llmToolSelectorMiddleware(options: LLMToolSelectorConfig) { | ||
| return createMiddleware({ | ||
| name: "LLMToolSelector", | ||
| contextSchema: LLMToolSelectorOptionsSchema, |
There was a problem hiding this comment.
We're missing here an option to always include some tools.
A user may want to make sure that some tools are always included (and these tools should be removed from the prompt; i.e., the model shouldn't necessarily know about them.)
| * }); | ||
| * ``` | ||
| */ | ||
| export function llmToolSelectorMiddleware(options: LLMToolSelectorConfig) { |
There was a problem hiding this comment.
I'd prefer renaming to toolSelection.ts personally and put some different strategies here in the long run... unless we think that bigTool.ts is obvious
| * Zod schema for tool selection structured output. | ||
| */ | ||
| const ToolSelectionSchema = z.object({ | ||
| selectedTools: z.array(z.string()).describe("List of selected tool names"), |
There was a problem hiding this comment.
Could we generate this dynamically so it's not a list of string, but a list of possible enums with descriptions?
3 participants
Overview
This PR introduces the
llmToolSelectorMiddleware- a new middleware for LangChain agents that intelligently selects the most relevant tools for each user query using an LLM-based strategy. This middleware helps reduce cognitive load on the main model and improves response quality by filtering tools before they reach the primary agent model.Features
Core Functionality
Usage Example
Benefits