[Question]: Why does FunctionAgent not generate a response after tool call with Anthropic LLM (but works with Ollama)?

[RakeshReddyKondeti]

Question Validation

• I have searched both the documentation and discord for an answer.

Question

Hi,

I'm using LlamaIndex's FunctionAgent with both Ollama LLM and Anthropic LLM. With Ollama, the agent calls the tool and then generates a response to the user as expected. With Anthropic LLM, the tool call is made successfully and the tool result is returned, but after that, the agent does not generate a follow-up response. There are no errors or warnings—the process just stops after the tool result.

What I’ve checked:

• The tool call and tool result are handled correctly.
• The same workflow works with Ollama LLM.
• No exceptions or errors are raised.
• Using llama-index-llms-anthropic v0.9.5.

Messages sent to Anthropic (printed just before the response object):

anthropic_messages, system_prompt = messages_to_anthropic_messages(messages, self.cache_idx)
# Printed anthropic_messages here

Example (anthropic_messages):

user- 'content': [
    {'type': 'text', 'text': 'Hello'}
]
assistant- 'content': [
    {'type': 'text', 'text': "Hello! I'm **AI Assistant**, your Digital Cooking Agent. \n\nHow can I help you today?"}
]
user- 'content': [
    {'type': 'text', 'text': 'I want to cook eggs today. how can i do that?'}
]
assistant- 'content': [
    {'id': 'toolu_01XpPXWym1Nu8W3XbrJEvBQf', 'input': {'cooking': {'to_be_cooked': 'eggs'}}, 'name': 'start_cooking', 'type': 'tool_use'}, 
    {'id': 'toolu_01TqML4SacyqDNzmh9M1s21H', 'input': {}, 'name': 'get_current_cooking_status', 'type': 'tool_use'}
]
user- 'content': [
    {'tool_use_id': 'toolu_01XpPXWym1Nu8W3XbrJEvBQf', 'type': 'tool_result', 'content': [{'type': 'text', 'text': "{'cooking': {'to_be_cooked': 'eggs', 'chilly': None, 'pepper': None, 'salt': None, 'oil': None}}"}]}, 
    {'tool_use_id': 'toolu_01TqML4SacyqDNzmh9M1s21H', 'type': 'tool_result', 'content': [{'type': 'text', 'text': "{'cooking': {'to_be_cooked': 'eggs', 'chilly': None, 'pepper': None, 'salt': None, 'oil': None}}"}]}
]

After this, the response object is not generated and nothing happens.

Is there something different about how Anthropic expects tool results to be formatted or streamed back? Any guidance would be appreciated!

[logan-markewich]

@RakeshReddyKondeti any way to reproduce would be appreciated here

[RakeshReddyKondeti]

Hi @logan-markewich here is a minimal reproducible example as requested.

import asyncio
from llama_index.core.agent.workflow import FunctionAgent
from llama_index.core.tools import FunctionTool
from llama_index.llms.anthropic import Anthropic
from llama_index.core.agent.workflow import (
    AgentStream,
    ToolCall,
    ToolCallResult,
)

def get_weather(location: str) -> str:
    return f"The current weather in {location} is sunny."

weather_tool = FunctionTool.from_defaults(
    name="get_weather",
    description="Get the current weather for a given location.",
    fn=get_weather,
)

llm = Anthropic(
    model="claude-sonnet-4-5",
    api_key="sk-...",  # <-- Replace with your key
    max_tokens=10000,
    thinking_dict={"type": "enabled", "budget_tokens": 1600}
)

agent = FunctionAgent(
    llm=llm,
    tools=[weather_tool],
    system_prompt="You are a helpful assistant that can provide weather information.",
)

async def main():
    while True:
        user_input = input("\n\nUser: ").strip()
        handler = agent.run(user_msg=user_input)
        async for ev in handler.stream_events():
            if isinstance(ev, AgentStream):
                print(ev.delta, end="", flush=True)
            elif isinstance(ev, ToolCall):
                print(f"\n\n>> Calling tool: {ev.tool_name} with kwargs: {ev.tool_kwargs}")
            elif isinstance(ev, ToolCallResult):
                print(f"\n\n>> Tool call result: {ev.tool_name} with result: {ev.tool_output}")

if __name__ == "__main__":
    asyncio.run(main())

Observed behavior:

• For questions that do not require tool calls, the agent responds as expected.
• For questions that require a tool call, the ToolCall is triggered and the tool output is generated successfully, but after the tool call, the agent does not generate any further response. It silently fails and does not return a follow-up message to the user.

Example output:

User: Hello
Hello! I'm here to help you with weather information. I can check the current weather for any location you'd like to know about. Just let me know which city or location you're interested in!

User: How is the weather today in Tokyo ?
{"location": "Tokyo"}

>> Calling tool: get_weather with kwargs: {'location': 'Tokyo'}

>> Tool call result: get_weather with result: The current weather in Tokyo is sunny.

User:

As shown above, after the tool call and result, the agent does not generate a final response.

[logan-markewich]

@RakeshReddyKondeti Really appreciate the example, this always helps a lot.

This works fine for me. Do you have the latest version of the anthropic integration?

Here's a google colab replicating with your code:
https://colab.research.google.com/drive/18Klu9nS1A6NAoR8bGnYIKGrs8_abTZt6?usp=sharing

However, I do spot the bug that is noted in this issue:
#20101

[RakeshReddyKondeti]

Hi @logan-markewich, thanks for sharing the Colab notebook. I think my issue was related to local dependencies. Once I updated my dependencies, it is working again. Closing this issue.

[RakeshReddyKondeti]

Hi @logan-markewich

I have a new problem now: when a tool call is triggered, the agent prints both the tool call arguments (e.g., {"location": "Tokyo"}) and the final response (e.g., The current weather in Tokyo is sunny.) in the user-facing output. Sometimes these outputs are intermixed, which makes the response confusing.

My observations:

• This behavior is also present in the Colab notebook you provided.
• For questions that do not require tool calls, the agent responds as expected.
• For questions that trigger a tool call, the tool arguments are printed along with the final answer.

Could you please clarify if I am doing something wrong in my usage, or if the llama-index API behaves differently across LLM frameworks? With the same code, Ollama LLM only prints the final user-facing response, but Anthropic LLM prints both the tool call arguments and the final response (sometimes with duplication).

Any guidance would be appreciated!

[logan-markewich]

This seems like this has always been the case for anthropic when reading the AgentStream event? But I agree its kind of weird, easy to fix