Skip to content

[smoke-detector] 🔁 Smoke Test Investigation - Smoke Copilot Run #18786022255: Permission Denied Recurrence #2301

New issue
New issue
Closed
Closed
[smoke-detector] 🔁 Smoke Test Investigation - Smoke Copilot Run #18786022255: Permission Denied Recurrence#2301
Labels
investigationsmoke-test
@github-actions

Description

@github-actions
github-actions
bot
opened on Oct 24, 2025

🔍 Smoke Test Investigation - Run #18786022255

Summary

The Smoke Copilot workflow failed with the SAME permission denial issue as #2288. Despite being closed, this issue has recurred 4 hours later on the same branch. The Copilot agent receives "Permission denied" when attempting to use safe-outputs MCP tools, even after commit e8bde4f added tools: ["*"] field to MCP configs.

Failure Details

  • Run: #18786022255
  • Commit: e8bde4f
  • Branch: copilot/update-copilot-agent-engine
  • Trigger: workflow_dispatch
  • Duration: 2.3 minutes
  • Failed Jobs: create_issue (7s duration)

Recurrence Timeline

Event Timestamp Details
First Occurrence 2025-10-24T12:03:06Z Run #18779136174
Issue #2288 Created 2025-10-24T12:14:18Z Investigation issue opened
Issue #2288 Closed 2025-10-24T12:29:43Z Closed by @pelikhan
Commit e8bde4f 2025-10-24T16:30:09Z Added tools:["*"] to MCP configs
Second Occurrence 2025-10-24T16:33:05Z Run #18786022255 (THIS RUN)

Time Between Occurrences: ~4.5 hours
Status: Issue recurred despite attempted fix

Root Cause Analysis

Primary Error (SAME AS #2288)

From /tmp/gh-aw/aw-mcp/logs/run-18786022255/agent-stdio.log/agent-stdio.log:13-14:

✗ create_issue
   Permission denied and could not request permission from user

Investigation Findings

MCP Server Status: ✅ WORKING CORRECTLY

  • Safe-outputs MCP server running properly
  • Tools registered: create_issue, missing_tool
  • Config parsed successfully

Agent Behavior: ✅ FOUND TOOLS, ❌ DENIED PERMISSION

  • Agent successfully listed merged PRs using GitHub MCP
  • Agent attempted to use safe-outputs-create_issue tool
  • Permission system blocked the tool call

Downstream Impact: ❌ CASCADING FAILURE

Error reading agent output file: ENOENT: no such file or directory, 
open '/tmp/gh-aw/safe-outputs/agent_output.json'

What Changed Between Occurrences

Commit e8bde4f: "Add tools field back to MCP configs with ["*"] value"

This commit attempted to fix MCP tool availability by:

  • Adding tools: ["*"] field to all MCP server configs
  • Using wildcard to allow all tools in Copilot CLI
  • Updating GitHub, Playwright, safe-outputs, and other MCP configs

Result: The permission denial still occurs despite this change.

Why the Fix Didn't Work

The tools: ["*"] field may configure which MCP tools are registered, but it does NOT address the Copilot CLI's tool authorization/permission system that's blocking tool usage.

This is NOT:

  • ❌ A tool registration issue (tools are registered)
  • ❌ A config parsing issue (config is correct)
  • ❌ An MCP server crash (server is running)

This IS:

  • ✅ A Copilot CLI authorization system issue
  • ✅ A tool permission/allowlist configuration problem
  • ✅ A potential Copilot CLI version compatibility issue

Failed Jobs and Errors

Job Sequence

  1. ✅ activation - succeeded (3s)
  2. ✅ agent - succeeded (58s) - Agent ran but couldn't use safe-outputs
  3. ✅ detection - succeeded (30s)
  4. ❌ create_issue - failed (7s)
  5. ⏭️ missing_tool - skipped

Error Summary

  • Total Errors: 7
  • Total Warnings: 4
  • Key Error: Permission denied for safe-outputs-create_issue

Pattern Information

  • Pattern ID: COPILOT_SAFE_OUTPUTS_PERMISSION_DENIED
  • Category: Permission/Authorization Error
  • Severity: High
  • Is Flaky: No - Consistent behavior
  • Total Occurrences: 2 (and counting)
  • First Seen: 2025-10-24T12:03:06Z
  • Last Seen: 2025-10-24T16:33:05Z

Comparison with Related Issues

Issue Problem Status Relation
#2288 Permission denied for safe-outputs Closed EXACT DUPLICATE
#2280 Malformed JSON config Closed (Fixed) Different - config parsing
#2281 Config treated as string array Closed (Fixed) Different - parsing bug

This is a RECURRENCE, not a new pattern.

Recommended Actions

Immediate (Critical Priority)

  • Reopen [smoke-detector] 🔍 Smoke Test Investigation - Smoke Copilot: Permission Denied for Safe-Outputs Tools #2288 or acknowledge this recurrence

    • Issue was closed but root cause remains unresolved
    • Why: Prevents issue from being marked as fixed when it's not

  • Investigate Copilot CLI authorization beyond MCP registration

    • The tools: ["*"] field did not resolve the issue
    • Check if there's a separate permission/allowlist mechanism
    • Why: Root cause is in Copilot CLI, not MCP config

  • Compare with Claude engine configuration

    • Claude smoke tests successfully use safe-outputs
    • Identify what Claude does differently
    • Why: Prove safe-outputs works with different engine

High Priority

  • Check Copilot CLI 0.0.349 release notes

    • Look for tool permission changes
    • Check for known issues with MCP tool authorization
    • Why: May be a version-specific bug

  • Test with explicit tool permissions in workflow

    # If Copilot supports workflow-level tool permissions:
permissions:
  safe-outputs:
    - create_issue
    - missing_tool
    • Why: May need explicit permission grants

  • Add pre-flight tool permission test

    • Check if safe-outputs tools are callable before running agent
    • Fail fast with clear error if permissions missing
    • Why: Better developer experience

Medium Priority

  • Engage Copilot CLI team

    • File bug report with Copilot CLI project
    • Provide reproduction case
    • Why: Issue is in Copilot CLI, not gh-aw

  • Consider workaround options

    • Use Claude engine for safe-outputs workflows until fixed
    • Implement alternative output mechanism for Copilot
    • Why: Unblock smoke tests

Prevention Strategies

  1. Don't close issues until verified fixed

    • Add verification step: re-run smoke test after fix
    • Wait 24 hours to ensure issue doesn't recur

  2. Automated recurrence detection

    • Pattern database now tracks occurrences
    • Smoke Detector will flag future recurrences

  3. Root cause validation

    • Ensure fixes address actual root cause, not symptoms
    • The tools: ["*"] fix addressed registration, not authorization

Technical Details

Environment

  • Copilot CLI: 0.0.349
  • Node.js: v24.10.0
  • Safe-Outputs MCP: v1.0.0
  • MCP Protocol: 2025-06-18

Agent Session

  • Task: Review last 5 merged PRs and create issue
  • Tools Available: GitHub MCP (✅ used), safe-outputs MCP (❌ denied)
  • Exit Code: 0 (success, but incomplete task)

MCP Config

{
  "tools": ["*"],
  "create_issue": {"max": 1, "min": 1},
  "missing_tool": {}
}

Historical Context

Related Investigations:

Pattern File: /tmp/gh-aw/cache-memory/patterns/copilot_safe_outputs_permission_denied.json

Fix Attempts:

  1. ✅ Fixed malformed JSON config ([smoke-detector] 🔍 Smoke Test Investigation - Smoke Copilot: Safe-Outputs MCP Crashes Due to Malformed Config JSON #2280)
  2. ✅ Fixed string-spreading bug ([smoke-detector] 🔍 Smoke Test Investigation - Smoke Copilot Run #18778382550: Safe-Outputs MCP Treats Config as Character Array #2281)
  3. ❌ Added tools: ["*"] field (e8bde4f) - DID NOT FIX ISSUE

Questions for Copilot Team

  1. How does Copilot CLI's tool permission system work?
  2. Is tools: ["*"] in MCP config sufficient to allow tool usage?
  3. Are there additional permission configurations needed?
  4. Is this a known issue in version 0.0.349?
  5. Why does Claude engine work with safe-outputs but Copilot doesn't?

Related Issues

Investigation Metadata:

AI generated by Smoke Detector - Smoke Test Failure Investigator

Metadata

Metadata

Assignees

No one assigned

    Labels

    investigationsmoke-test

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions