[2025-07-28] Incident Thread

[github-actions[bot]]

❗ An incident has been declared:

Incident with Issues, API Requests and Pull Requests

Subscribe to this Discussion for updates on this incident. Please upvote or emoji react instead of commenting +1 on the Discussion to avoid overwhelming the thread. Any account guidance specific to this incident will be shared in thread and on the Incident Status Page.

[github-actions[bot]]

Update

Some GitHub services are experiencing degraded performance. Team is currently investigating to determine a cause and mitigation.

[github-actions[bot]]

Update

Some GitHub services continue to experience degraded performance. Team is looking into networking issues. We will continue to keep users updated on progress towards mitigation.

[github-actions[bot]]

Update

Approximately 4% of requests to impacted services continue to error. The team is continuing its work to mitigate this incident.

[github-actions[bot]]

Update

Team is deploying a mitigation for this incident. We will update again once we have verified the fix.

[github-actions[bot]]

Update

Mitigation has deployed. We are seeing recovery across all impacted services.

[github-actions[bot]]

Incident Resolved

This incident has been resolved.

[github-actions[bot]]

Incident Summary

Between July 28, 2025, 22:23:00 UTC and July 29, 2025 02:06:00 UTC, GitHub experienced degraded performance across multiple services including API, Issues, GraphQ and Pull Requests. During this time, approximately 4% of Web and API requests resulted in 500 errors.

This incident was caused by DNS resolution failure while decommissioning infrastructure hosts. We resolved the incident by removing references to the stale hosts.

We are working to improve our host replacement process by correcting our automatic host ejection behavior and by ensuring configuration is updated before hosts are decommissioned. This will prevent similar issues in the future.