Enhance Qurator context with automatic README.md loading

[nl0]

Summary

• Enhance Qurator assistant with automatic context file loading from directory hierarchy
• Support both README.md and AGENTS.md files throughout bucket/package structure
• Implement smart limits: 10KB file size, 10 non-root file maximum
• Add package metadata context with system/user separation

Key Features

Context File Loading

• Loads both README.md and AGENTS.md files from directory hierarchy
• AGENTS.md has priority over README.md when file limit is reached
• Works for bucket browsing, file viewing, and package navigation
• 10KB size limit per file (reduced from 100KB for better performance)
• 10 file limit for non-root files (root files always included)
• Files loaded from most specific (current) to least specific (root)

Package Context

• System Metadata (<package-info> tag): bucket, name, hash, modified, message, workflow, stats
• User Metadata (<package-metadata> tag): custom userMeta when present
• Separate contexts for package root and package directories

Technical Improvements

• De-emphasized tool UI: Collapsible tool messages with icons instead of text

Context Markers

• bucketContextFilesReady - Bucket root context loaded
• dirContextFilesReady - Directory hierarchy loaded
• fileContextFilesReady - File parent context loaded
• packageMetadataReady - Package metadata available
• packageContextFilesReady - Package root context loaded
• packageDirContextFilesReady - Package directory context loaded

Testing

Verify context loading by checking Qurator DevTools for the above markers and inspecting assistant context messages.

[codecov]

Codecov Report

❌ Patch coverage is 3.38983% with 114 lines in your changes missing coverage. Please review.
✅ Project coverage is 39.32%. Comparing base (6c50227) to head (ddd4b59).

Files with missing lines	Patch %	Lines
...log/app/components/Assistant/Model/ContextFiles.ts	0.00%	67 Missing ⚠️
...containers/Bucket/PackageTree/AssistantContext.tsx	0.00%	20 Missing and 2 partials ⚠️
...log/app/containers/Bucket/File/AssistantContext.ts	0.00%	5 Missing ⚠️
catalog/app/utils/XML.ts	16.66%	4 Missing and 1 partial ⚠️
catalog/app/containers/Bucket/AssistantContext.tsx	0.00%	4 Missing ⚠️
.../app/containers/Bucket/PackageTree/PackageTree.tsx	0.00%	4 Missing ⚠️
...talog/app/containers/Bucket/DirAssistantContext.ts	0.00%	3 Missing ⚠️
catalog/app/utils/LogicalKeyResolver.tsx	40.00%	3 Missing ⚠️
catalog/app/containers/Bucket/Bucket.tsx	0.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #4561      +/-   ##
==========================================
- Coverage   39.44%   39.32%   -0.12%     
==========================================
  Files         852      855       +3     
  Lines       36860    36975     +115     
  Branches     5764     6037     +273     
==========================================
+ Hits        14538    14542       +4     
+ Misses      21804    21206     -598     
- Partials      518     1227     +709     

Flag	Coverage Δ
api-python	91.67% <ø> (ø)
catalog	20.11% <3.38%> (-0.08%)	⬇️
lambda	92.60% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[greptile-apps]

Greptile Overview

Summary

This PR enhances Qurator's context system by automatically loading README.md files from directory hierarchies to provide better domain awareness.

Key Changes

• New ContextFiles module: Core functionality for loading README.md files from S3 with proper error handling and 100KB truncation
• Hierarchical context loading: README files are loaded from current location up to bucket root, providing contextual information at each level
• Integration across views: Added README context to bucket root, directory browsing, and file viewing
• Graceful error handling: Missing README files are handled silently without breaking functionality

Technical Implementation

The implementation follows existing patterns using LazyContext providers and Effect library. README files are loaded in parallel using Promise.all() and formatted as XML messages for the assistant. The system includes appropriate loading states and markers for context readiness tracking.

Confidence Score: 4/5

• This PR is largely safe to merge with some minor concerns
• Score reflects well-structured implementation following existing patterns, proper error handling, and thorough integration. Minor deduction for potential undefined body check and CI deployment concerns
• Pay attention to ContextFiles.ts for the undefined body handling and verify CI deployment intentions

Important Files Changed

File Analysis

Filename	Score	Overview
catalog/app/components/Assistant/Model/ContextFiles.ts	4/5	New core module for README.md loading with proper error handling and truncation
catalog/app/containers/Bucket/AssistantContext.tsx	5/5	New bucket context provider that loads README.md from bucket root
catalog/app/containers/Bucket/DirAssistantContext.ts	5/5	Enhanced directory context with README hierarchy loading from current path upward
catalog/app/containers/Bucket/File/AssistantContext.ts	5/5	Enhanced file context with parent directory README loading
.github/workflows/deploy-catalog.yaml	3/5	Added qctx branch trigger and commented out GovCloud/MP deployments

Sequence Diagram

sequenceDiagram
    participant User
    participant BucketComponent as Bucket.tsx
    participant DirComponent as Dir.tsx
    participant FileComponent as File.js
    participant BucketContext as BucketContext
    participant DirContext as DirContextFiles
    participant FileContext as FileContextFiles
    participant ContextFiles as ContextFiles.ts
    participant S3
    participant Assistant

    User->>BucketComponent: Navigate to bucket
    BucketComponent->>BucketContext: Initialize with bucket name
    BucketContext->>ContextFiles: loadContextFile(s3, bucket, '')
    ContextFiles->>S3: getObject(bucket, 'README.md')
    S3-->>ContextFiles: README.md content or 404
    ContextFiles-->>BucketContext: ContextFileContent or null
    BucketContext->>Assistant: Format as XML message
    
    User->>DirComponent: Navigate to directory
    DirComponent->>DirContext: Initialize with bucket, path
    DirContext->>ContextFiles: loadContextFileHierarchy(s3, bucket, path, '')
    ContextFiles->>ContextFiles: buildPathChain(path, stopAt='')
    loop For each path in chain
        ContextFiles->>S3: getObject(bucket, path + '/README.md')
        S3-->>ContextFiles: README.md content or 404
    end
    ContextFiles-->>DirContext: Array of ContextFileContent
    DirContext->>Assistant: Format as XML messages
    
    User->>FileComponent: View file
    FileComponent->>FileContext: Initialize with bucket, path
    FileContext->>ContextFiles: Get parent directory path
    FileContext->>ContextFiles: loadContextFileHierarchy(s3, bucket, parentPath, '')
    ContextFiles->>ContextFiles: buildPathChain(parentPath, stopAt='')
    loop For each parent path
        ContextFiles->>S3: getObject(bucket, path + '/README.md')
        S3-->>ContextFiles: README.md content or 404
    end
    ContextFiles-->>FileContext: Array of ContextFileContent
    FileContext->>Assistant: Format as XML messages

    Note over Assistant: Now has context from README hierarchy for better assistance

Loading

_{11 files reviewed, 2 comments}

_{Edit Code Review Agent Settings | Greptile}