Skip to content

Conversation

priyansh4320
Copy link
Collaborator

@priyansh4320 priyansh4320 commented Sep 17, 2025

Depends on #2097

Why are these changes needed?

Extending DocAgent Beyond Vector RAG to Graph RAG

The current DocAgent implementation in ag2 relies solely on vector-based retrieval-augmented generation (RAG), which while effective for semantic similarity searches, has inherent limitations in capturing complex relational information and multi-hop reasoning scenarios. Vector RAG excels at finding documents with similar embeddings but struggles with queries that require understanding relationships between entities, temporal sequences, or hierarchical structures within the knowledge base. By extending DocAgent to support Graph RAG, you could leverage knowledge graphs to represent documents as interconnected nodes and relationships, enabling more sophisticated reasoning patterns like "find all research papers that cite work by authors who collaborated with X" or "trace the evolution of a concept across multiple related documents."

Summary

  • Updated DocAgent for Neo4j Graph Rag Capabilities.
  • Updated Configuration for Dynamic RAG selection with a new parameter called rag_config.
  • added result aggregator for when both RAGs are enabled.
  • Sample Configuration for Dynamic RAG
llm_config = LLMConfig(
    model="gpt-5-nano",
    api_key=os.getenv("OPENAI_API_KEY"),
    api_type="openai",
)

doc_agent = DocAgent(
    llm_config=llm_config,
    rag_config={
        "graph": {
            "host": "bolt://localhost",
            "port": 7687,
            "username": "neo4j",
            "password": "admin1234",
        },
        "vector": {"engine": VectorChromaQueryEngine(collection_name="test7")},
    },
)

res = doc_agent.run(
    message="ingest the file test/agentchat/contrib/graph_rag/research_paper.pdf also ingest test/agentchat/contrib/graph_rag/Toast_financial_report.pdf? and who are theauthor of the research paper?",
    max_turns=1,
)
res.process()
Screenshot 2025-09-22 at 8 18 03 PM

Updates on the design
Untitled diagram _ Mermaid Chart-2025-09-22-165128

Related issue number

#2113

Checks

@priyansh4320 priyansh4320 added enhancement New feature or request agents:docagent Document Agent labels Sep 20, 2025
@priyansh4320 priyansh4320 force-pushed the Dynamic-rag-on-DocAgent-ocr-support branch from f1daca9 to f835f50 Compare September 22, 2025 01:54
@priyansh4320 priyansh4320 self-assigned this Sep 22, 2025
@priyansh4320 priyansh4320 force-pushed the Dynamic-rag-on-DocAgent-ocr-support branch 2 times, most recently from 94ec5b6 to e082739 Compare September 23, 2025 00:11
@priyansh4320 priyansh4320 marked this pull request as ready for review September 23, 2025 00:30
@priyansh4320 priyansh4320 force-pushed the Dynamic-rag-on-DocAgent-ocr-support branch from e082739 to 444b5fb Compare September 23, 2025 18:20
@marklysze
Copy link
Collaborator

@bassilkhilo-ag2 can you please try this out

@priyansh4320 priyansh4320 force-pushed the Dynamic-rag-on-DocAgent-ocr-support branch from 3beebd3 to 1ba34fa Compare September 26, 2025 00:38
@priyansh4320 priyansh4320 force-pushed the Dynamic-rag-on-DocAgent-ocr-support branch from 1ba34fa to 576606a Compare October 3, 2025 23:41
@priyansh4320 priyansh4320 force-pushed the Dynamic-rag-on-DocAgent-ocr-support branch from 576606a to ef109ad Compare October 3, 2025 23:49
@codecov
Copy link

codecov bot commented Oct 3, 2025

Codecov Report

❌ Patch coverage is 40.92072% with 231 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...agents/experimental/document_agent/task_manager.py 14.63% 140 Missing ⚠️
...ents/experimental/document_agent/document_agent.py 8.45% 65 Missing ⚠️
.../experimental/document_agent/task_manager_utils.py 87.24% 13 Missing and 6 partials ⚠️
...agents/experimental/document_agent/parser_utils.py 0.00% 7 Missing ⚠️
Files with missing lines Coverage Δ
...agents/experimental/document_agent/parser_utils.py 23.61% <0.00%> (-71.85%) ⬇️
.../experimental/document_agent/task_manager_utils.py 87.24% <87.24%> (ø)
...ents/experimental/document_agent/document_agent.py 25.66% <8.45%> (+4.58%) ⬆️
...agents/experimental/document_agent/task_manager.py 14.63% <14.63%> (ø)

... and 40 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents:docagent Document Agent enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants