Skip to content

Conversation

@ehutt
Copy link
Contributor

@ehutt ehutt commented Oct 10, 2025

Changes Include:

  • dataframe eval methods return json instead of string in score/details columns
  • adds a default tqdm progress bar formatter to dataframe evals + flag to hide
  • moves LLM to a top level import since it is commonly used with Evaluators
  • updates notebooks in tutorials accordingly with the new import statements

Bug fix:

  • runningbind_evaluator on an LLMEvaluator returns an error due to attempting to deep copy the LLM object which can't be pickled. Now the method does a shallow copy of the evaluator and only deep copies the input_mapping, which is the only property we don't want the original and copy to share.

Note

Switch dataframe eval outputs to dicts, add default/hideable tqdm progress bar, expose LLM at top level, and fix bind_evaluator copying; update notebooks/tests accordingly.

  • Evals 2.0 (Core):
    • Return JSON-serializable dicts (not strings) in {score}_score and {evaluator}_execution_details columns for dataframe evals.
    • Add progress bar controls: default formatter via default_tqdm_progress_bar_formatter(...) and hide_tqdm_bar flag for sync/async dataframe evals.
    • Fix bind_evaluator(...): use shallow copy of evaluator (avoids deepcopying LLM with locks) and deep copy only the input_mapping.
  • API/Exports:
    • Expose LLM at top-level import (from phoenix.evals import LLM).
  • Utils:
    • Enhance to_annotation_dataframe to parse dict-backed scores; export default_tqdm_progress_bar_formatter.
  • Docs/Tutorials/Examples:
    • Update imports to use top-level LLM; minor notebook cleanup.
  • Tests:
    • Adjust expectations to handle dict outputs for scores and execution details; add progress bar coverage.

Written by Cursor Bugbot for commit 6d7c3f0. This will update automatically on new commits. Configure here.

@ehutt ehutt requested review from a team as code owners October 10, 2025 23:56
@github-project-automation github-project-automation bot moved this to 📘 Todo in phoenix Oct 10, 2025
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.


Powered by ReviewNB

@dosubot dosubot bot added the size:XXL This PR changes 1000+ lines, ignoring generated files. label Oct 10, 2025
@github-project-automation github-project-automation bot moved this from 📘 Todo to 👍 Approved in phoenix Oct 20, 2025
@ehutt ehutt merged commit 5546179 into main Oct 22, 2025
50 checks passed
@ehutt ehutt deleted the ehutt/random-fixes branch October 22, 2025 18:41
@github-project-automation github-project-automation bot moved this from 👍 Approved to ✅ Done in phoenix Oct 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL This PR changes 1000+ lines, ignoring generated files.

Projects

Status: ✅ Done

Development

Successfully merging this pull request may close these issues.

2 participants