Skip to content

Conversation

casparb
Copy link
Collaborator

@casparb casparb commented Aug 25, 2025

This PR adds error handling and retry mechanisms for create_agent structured output via a handle_errors parameter in ToolOutput.

Changes:

  • Adds MultipleStructuredOutputsError exception for when models incorrectly call multiple structured output tools simultaneously
  • Adds StructuredOutputParsingError exception for when tool arguments fail to parse according to the schema
  • Implements automatic error handling logic that re-prompts the model with a configurable error message when structured output failures occur via handle_errors policy in ToolOutput:
class ToolOutput:
    ...
    handle_errors: Union[
        bool,                         # True: retry all, False: no retry
        str,                          # Custom static error message for all errors
        type[Exception],              # Retry only this exception type
        tuple[type[Exception], ...],  # Retry only these exception types
        Callable[[Exception], str],   # Custom callable returning error message
    ]
    """Error handling strategy. Default: True (retry on all error types with default error message)"""

Examples:

# Retry all errors
ToolOutput(WeatherReport)

# No retry
ToolOutput(WeatherReport, handle_errors=False)

# Custom message for all errors
ToolOutput(WeatherReport, handle_errors="Please provide valid data")

# Only retry specific error type
ToolOutput(WeatherReport, handle_errors=StructuredOutputParsingError)

# Multiple error types
ToolOutput(WeatherReport, handle_errors=(MultipleStructuredOutputsError, StructuredOutputParsingError))

# Custom logic
ToolOutput(
    Union[WeatherReport, LocationInfo],
    handle_errors=lambda e: "Only one response please" if isinstance(e, MultipleStructuredOutputsError) else "Invalid format"
)

Copy link

vercel bot commented Aug 25, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Preview Comments Updated (UTC)
langgraph-docs-preview Ignored Ignored Preview Aug 26, 2025 0:59am

@casparb casparb changed the base branch from main to eugene/update_agent_implementation August 25, 2025 05:50
Copy link
Collaborator

@sydney-runkle sydney-runkle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

By default, we want to retry when a model calls multiple structured output tools right?

One other thing I want to think about (can be in my PR), we might want to support this in ToolNode as well, so like AgentRetryPolicy might be a better name...

@sydney-runkle
Copy link
Collaborator

Ah one other follow up - why introduce the cognitive overhead of a retry dataclass right now?

should we just do a callable? that offers the same impact and is less complex

@sydney-runkle sydney-runkle marked this pull request as ready for review August 25, 2025 17:42
Copy link
Collaborator

@sydney-runkle sydney-runkle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work on testing.

Please update description of PR + docstrings as well!

@casparb casparb force-pushed the caspar/structured_output_error_handling branch from c474cda to d48d02f Compare August 25, 2025 21:04
@sydney-runkle sydney-runkle merged commit cf615a4 into eugene/update_agent_implementation Aug 26, 2025
45 of 46 checks passed
@sydney-runkle sydney-runkle deleted the caspar/structured_output_error_handling branch August 26, 2025 13:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants