Skip to content

Conversation

@miguelg719
Copy link
Collaborator

why

what changed

test plan

@changeset-bot
Copy link

changeset-bot bot commented Sep 25, 2025

⚠️ No Changeset found

Latest commit: 6c520e2

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Overview

Summary

This PR comprehensively updates the evaluation CLI documentation to reflect the new standalone CLI interface. The changes replace outdated npm-based commands with the new evals CLI syntax throughout both user-facing documentation and developer README files.

Key Changes:

  • Updated command syntax from npm run evals to evals run across all documentation
  • Added detailed CLI installation instructions with step-by-step setup guide
  • Introduced external benchmark support documentation (WebBench, GAIA, WebVoyager, etc.)
  • Added visual CLI help screenshot to improve user experience
  • Streamlined eval categories from 3 to 2 types (Deterministic and LLM-based)
  • Enhanced command examples with proper flag usage and configuration options

The documentation now provides a much clearer onboarding experience for developers wanting to use Stagehand's evaluation system with consistent, accurate command references throughout.

Confidence Score: 5/5

  • This PR is completely safe to merge with no risk
  • Documentation-only changes with no code modifications, consistent command syntax updates, and improved user experience through clear instructions and visual aids
  • No files require special attention

Important Files Changed

File Analysis

Filename        Score        Overview
docs/configuration/evals.mdx 5/5 Comprehensive documentation update replacing old npm-based commands with new CLI interface, adding installation steps, command examples, and external benchmark support
docs/media/evals-cli.png 5/5 New CLI help screenshot showing clean command interface with commands (run, list, config, help) and usage syntax
evals/README.md 5/5 Updated README with consistent CLI command examples, removing 'pnpm' prefix and showing proper evals command usage throughout

Sequence Diagram

sequenceDiagram
    participant User as Developer
    participant Docs as Documentation
    participant CLI as Evals CLI
    participant Config as evals.config.json
    
    Note over User,Config: Documentation Update Flow
    
    User->>Docs: Read updated evals documentation
    Docs-->>User: Shows new CLI commands (evals run, evals config)
    
    User->>CLI: Install CLI (pnpm run build:cli)
    CLI-->>User: CLI installed successfully
    
    User->>CLI: evals help
    CLI-->>User: Shows command structure and options
    
    User->>CLI: evals run act -e browserbase -t 5
    CLI->>Config: Load eval configuration
    Config-->>CLI: Returns eval tasks and settings
    CLI-->>User: Executes evaluations with specified parameters
    
    Note over User,Config: Key Changes in PR
    Note over Docs: - Updated command syntax from npm to evals CLI
    Note over Docs: - Added installation instructions
    Note over Docs: - Added external benchmark support
    Note over Docs: - Included CLI screenshot
Loading

2 files reviewed, no comments

Edit Code Review Agent Settings | Greptile

@miguelg719 miguelg719 merged commit 108de3c into main Sep 26, 2025
15 checks passed
miguelg719 added a commit that referenced this pull request Oct 2, 2025
# why

# what changed

# test plan
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants