Skip to content

Conversation

ahgraber
Copy link
Contributor

Fixes #1538

  • Add final instruction for better json schema adherence and reduce "filler" text
  • Add extract_json function to identify json by pairs of [] or {}
  • Add unit tests for extract_json

Notes:
In my testing, these changes vastly reduce the number of times the 'repair' agent is triggered. However, this does not fix all parse errors:

  1. In some cases, json is valid json, but keys are invalid
  2. In some cases, especially with more complex Pydantic models (e.g., Themes, which is composed of List[Theme]), the Pydantic json schema includes $defs. As a result, some LLMs will return $defs or include "$schema": "https://json-schema.org/draft/...; either of these additional keys will cause the Pydantic parser to fail.

@dosubot dosubot bot added the size:L This PR changes 100-499 lines, ignoring generated files. label Oct 20, 2024
@shahules786 shahules786 requested a review from jjmachan October 21, 2024 14:06
Copy link
Member

@jjmachan jjmachan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks a lot @ahgraber for this PR ❤️
will be releasing it today

@jjmachan jjmachan merged commit 8deca92 into explodinggradients:main Oct 22, 2024
16 checks passed
@ahgraber ahgraber deleted the fix/output-json-parsing branch October 26, 2024 15:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expensive agentic failure loop

2 participants