Fix Copilot log parser: accumulate token usage and improve JSON block detection

[Copilot]

Problem

The Copilot JavaScript log parser had two issues with the new debug log format:

1. Token statistics not extracted: Only the last API response's token usage was shown, not the cumulative total across all responses
2. Multiple responses parsed incorrectly: JSON block detection bug caused sequential API responses to be merged or skipped

Changes

Token Usage Accumulation

Modified parseDebugLogFormat() to accumulate token usage across all API responses:

// Before: Only kept last response's usage
entries._lastResult = {
  type: "result",
  usage: jsonData.usage,  // Replaces previous usage
};

// After: Accumulates across all responses
if (!entries._accumulatedUsage) {
  entries._accumulatedUsage = { input_tokens: 0, output_tokens: 0 };
}
entries._accumulatedUsage.input_tokens += jsonData.usage.prompt_tokens;
entries._accumulatedUsage.output_tokens += jsonData.usage.completion_tokens;

Also normalized OpenAI's field names (prompt_tokens/completion_tokens) to the standard format (input_tokens/output_tokens) used by the renderer.

JSON Block Detection Fix

Fixed bug where lines with timestamps weren't properly ending JSON blocks. The parser now distinguishes JSON continuation lines from log entry lines:

// Check if cleaned line is JSON content
const cleanLine = line.replace(/^\d{4}-\d{2}-\d{2}T[\d:.]+Z \[DEBUG\] /, "");
const isJsonContent = /^[{\[}\]"]/.test(cleanLine);

if (!isJsonContent) {
  // End block and process accumulated JSON
} else {
  // Continue accumulating JSON lines
}

This ensures lines like 2025-10-21T01:00:02.000Z [DEBUG] response (Request-ID test-2): properly end the current JSON block, while lines like 2025-10-21T01:00:00.500Z [DEBUG] { are correctly identified as JSON content.

Impact

• ✅ Token usage now shows correct cumulative totals across entire workflow execution
• ✅ Turn counts accurately reflect multiple API interactions
• ✅ Streaming debug logs with multiple responses parse correctly

Testing

All 29 tests passing, including new test for token accumulation across multiple responses.

Note

Tool call response content parsing (showing ✅/❌ instead of ❓ for tool results) requires access to actual workflow run logs to understand the exact format. The infrastructure is ready to support this once we can examine real log samples from the new Copilot CLI version.

Original prompt

Update the copilot javascript log parser with the new log format. In particular,

• the parsed tool calls are missing the reponse content
• the total token stats are not extracted

Here is a run https://github.com/githubnext/gh-aw/actions/runs/18668628184/job/53225071044

You can use ./gh-aw audit 18668628184 to download the run and the artifacts.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[github-actions]

Agentic Changeset Generator triggered by this pull request.