This project implements a sophisticated multi-agent system to tackle complex reasoning and task-solving challenges, specifically targeting the GAIA benchmark. It leverages the smolagents library to create a team of specialized AI agents that collaborate to answer questions and complete tasks.

The core architecture follows a manager-worker pattern:

• A Manager Agent oversees the entire process, breaking down complex questions into smaller, manageable sub-tasks.
• A Retrieval Expert Agent specializes in gathering information from various sources, including web search, Wikipedia, Arxiv, and GitHub.
• A Logic Expert Agent excels at programming, logical reasoning, and mathematical calculations, using the information provided to derive final answers.
• A Browser Expert Agent excels at gathering web information through browser-use style with playwright mcp.

• Multi-Agent System: Utilizes a hierarchical team of agents for robust problem-solving.
• Extensible Toolset: Integrates a wide array of tools for web crawling, document parsing (PDF, DOCX, HTML), media processing (images, audio, YouTube), and scientific data analysis.
• Code-First Agents: Agents generate and execute Python code to perform actions, enabling complex logic and dynamic tool use.
• GAIA Benchmark Ready: Designed to run and evaluate tasks from the GAIA dataset out-of-the-box.
• Observability: Integrated with Phoenix/OpenInference for detailed tracing and monitoring of agent behavior.

Follow these steps to set up and run the GAIA-Agent system.

• Python 3.9+
• An environment variable file (.env) with your API keys:

  OPENAI_API_KEY=
  OPENAI_BASE_URL=
  
  MINERU_BASE_URL=https://mineru.net/api/v4
  MINERU_API_TOKEN=
  
  SERPAPI_API_KEY=
  
  PHOENIX_API_KEY=
  PHOENIX_COLLECTOR_ENDPOINT=
  
  GITHUB_API_TOKEN=
  
  GOOGLE_API_KEY=

1. Clone the repository:

   git clone <repository_url>
   cd <repository_directory>

2. Install the required Python packages:

   pip install -r requirements.txt

3. Download the GAIA dataset and place it in the data/ directory as instructed on the Hugging Face dataset page. Your directory structure should look like this:

   data/
   └── gaia/
   

Execute the main script to start processing tasks from the GAIA dataset. Each task runs in a dedicated workspace to isolate its files and logs.

python run_gaia.py --run_name "my_first_run" --concurrency 4

Command-Line Arguments:

• --run_name: A unique name for your execution run. Outputs will be saved under this name. (Default: gaia_run)
• --concurrency: The number of tasks to run in parallel. (Default: 10)
• --split: The dataset split to use (validation or test). (Default: validation)
• --use_phoenix: (Flag) Enable to send traces to Phoenix for monitoring.

.
├── data/                 # GAIA dataset files
├── logs/                 # Execution logs for each task
├── output/               # Raw and processed results from agent runs
├── prompts/              # YAML prompt templates for the agents
├── smolagents/           # The smolagents library submodule
├── src/                  # Core source code
│   ├── tools/            # Custom tools for the agents
│   ├── extractor.py      # Extracts final answers from agent output
│   └── utils.py          # Utility functions
├── workspaces/           # Isolated directories for each task run
├── evaluate_result.py    # Script to score the agent's predictions
├── requirements.txt      # Project dependencies
└── run_gaia.py           # Main script to launch the agent system

After a run completes, you can evaluate the performance of the agents against the ground truth answers. The run_gaia.py script automatically attempts to extract the final answer from the agent's conversation history.

To calculate the final scores, run the evaluation script:

python evaluate_result.py

This will process the output files located in output/<split>/<run_name>.jsonl and generate a report with accuracy and other metrics.

This project is built upon the smolagents library, which provides the foundation for creating code-first, tool-augmented agents. The core logic for the agents, their interaction loops, and tool integration is handled by this powerful and lightweight framework. See the smolagents/ directory for more details.

The behavior and personality of each agent are defined by prompt templates located in the prompts/ directory. These YAML files structure the system prompts, planning instructions, and task-specific guidelines for the Manager, Retrieval, and Logic agents.

The agents are equipped with a wide range of tools located in src/tools/. These are standard Python functions that agents can call to interact with the outside world. Below is a detailed list of the available tools: