Get email notifications whenever GitHub creates, updates or resolves an incident.
Get text message notifications whenever GitHub creates or resolves an incident.
Get incident updates and maintenance status messages in Slack.
Subscribe via Slack
By subscribing you acknowledge our Privacy Policy. In addition, you agree to the Atlassian Cloud Terms of Service and acknowledge Atlassian's Privacy Policy.
Get webhook notifications whenever GitHub creates an incident, updates an incident, resolves an incident or changes a component status.
Visit our support site.
All Systems Operational
Git Operations
?
Operational
API Requests
?
Operational
Webhooks
?
Operational
Visit www.githubstatus.com for more information
Operational
Issues
?
Operational
Pull Requests
?
Operational
Actions
?
Operational
Packages
?
Operational
Pages
?
Operational
Codespaces
?
Operational
Copilot
Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Past Incidents
Jul 17, 2024
No incidents reported today.
Jul 16, 2024
Resolved -
This incident has been resolved.
Jul 16, 03:07 UTC
Jul 16, 03:07 UTC
Update -
Copilot chat is error rates are improving and we are continuing to monitor system health.
Jul 16, 02:56 UTC
Jul 16, 02:56 UTC
Update -
Copilot chat is experiencing elevated error rates. We have identified the root cause and are working on remediation.
Jul 16, 02:10 UTC
Jul 16, 02:10 UTC
Update -
Copilot chat is experiencing elevated error rates and we are currently investigating the issue.
Jul 16, 01:35 UTC
Jul 16, 01:35 UTC
Investigating -
We are currently investigating this issue.
Jul 16, 00:53 UTC
Jul 16, 00:53 UTC
Jul 15, 2024
No incidents reported.
Jul 14, 2024
No incidents reported.
Jul 13, 2024
Resolved -
This incident has been resolved.
Jul 13, 19:27 UTC
Jul 13, 19:27 UTC
Update -
Copilot is operating normally.
Jul 13, 19:26 UTC
Jul 13, 19:26 UTC
Update -
Our upstream provider continues to recover and we expect services to return to normal as more progress is made. We will provide another update by 20:00 UTC.
Jul 13, 18:01 UTC
Jul 13, 18:01 UTC
Update -
Our upstream provider is making good progress recovering and we are validating that services are nearing normal operations. We will provide another update by 18:00 UTC.
Jul 13, 16:09 UTC
Jul 13, 16:09 UTC
Update -
Our upstream provider is gradually recovering the service. We will provide another update at 23:00 UTC.
Jul 13, 11:18 UTC
Jul 13, 11:18 UTC
Update -
We are continuing to wait on our upstream provider to see full recovery. We will provide another update at 11:00 UTC
Jul 13, 03:50 UTC
Jul 13, 03:50 UTC
Update -
The error rate for Copilot chat requests remains steady at less than 10%. We are continuing to investigate with our upstream provider.
Jul 13, 03:20 UTC
Jul 13, 03:20 UTC
Update -
Copilot is experiencing degraded performance. We are continuing to investigate.
Jul 13, 02:20 UTC
Jul 13, 02:20 UTC
Update -
We have applied several mitigations to Copilot chat, reducing errors to less than 10% of all chat requests. We are continuing to investigate the issue with our upstream provider.
Jul 13, 02:19 UTC
Jul 13, 02:19 UTC
Update -
Copilot chat is experiencing degraded performance, impacting up to 60% of all chat requests. We are continuing to investigate the issue with our upstream provider.
Jul 13, 01:32 UTC
Jul 13, 01:32 UTC
Update -
Copilot chat is currently experiencing degraded performance, impacting up to 60% of all chat requests. We are investigating the issue.
Jul 13, 00:49 UTC
Jul 13, 00:49 UTC
Update -
Copilot is experiencing degraded availability. We are continuing to investigate.
Jul 13, 00:29 UTC
Jul 13, 00:29 UTC
Update -
Copilot API chat experiencing significant failures to backend services
Jul 13, 00:18 UTC
Jul 13, 00:18 UTC
Investigating -
We are investigating reports of degraded performance for Copilot
Jul 13, 00:18 UTC
Jul 13, 00:18 UTC
Jul 12, 2024
No incidents reported.
Jul 11, 2024
Resolved -
This incident has been resolved.
Jul 11, 15:21 UTC
Jul 11, 15:21 UTC
Update -
Copilot is operating normally.
Jul 11, 15:21 UTC
Jul 11, 15:21 UTC
Update -
We have mitigated the intermittent timeout errors impacting Copilot’s Chat functionality and expect the incident to be resolved shortly.
Jul 11, 15:19 UTC
Jul 11, 15:19 UTC
Update -
We continue to investigate the cause of intermittent timeouts impacting Copilot’s Chat functionality. This is impacting a small fraction of customers. The timeout errors we are seeing has reduced back to healthy levels for the last 60 minutes but we are monitoring closely.
Jul 11, 15:04 UTC
Jul 11, 15:04 UTC
Update -
We continue to investigate the cause of intermittent timeouts impacting Copilot’s Chat functionality. This is impacting a small fraction of customers. We will provide further updates as we continue resolving the issue.
Jul 11, 14:14 UTC
Jul 11, 14:14 UTC
Update -
We continue to investigate the cause of intermittent timeouts impacting Copilot’s Chat functionality. This is impacting a small fraction of customers. We will provide further updates as we continue resolving the issue.
Jul 11, 13:32 UTC
Jul 11, 13:32 UTC
Update -
Copilot's Chat functionality is experiencing intermittent timeouts, we are investigating the issue.
Jul 11, 13:02 UTC
Jul 11, 13:02 UTC
Investigating -
We are investigating reports of degraded performance for Copilot
Jul 11, 13:02 UTC
Jul 11, 13:02 UTC
Jul 10, 2024
No incidents reported.
Jul 9, 2024
No incidents reported.
Jul 8, 2024
Resolved -
On July 8th, 2024, between 18:18 UTC and 19:11 UTC, various services relying on static assets were degraded, including user uploaded content on github.com, access to docs.github.com and Pages sites, and downloads of Release assets and Packages.
The outage primarily affected users in the vicinity of New York City, USA, due to a local CDN disruption.
Service was restored without our intervention.
We are working to improve our external monitoring, which failed to detect the issue and will be evaluating a backup mechanism to keep critical services available, such as being able to load assets on GitHub.com, in the event of an outage with our CDN.
Jul 8, 19:45 UTC
The outage primarily affected users in the vicinity of New York City, USA, due to a local CDN disruption.
Service was restored without our intervention.
We are working to improve our external monitoring, which failed to detect the issue and will be evaluating a backup mechanism to keep critical services available, such as being able to load assets on GitHub.com, in the event of an outage with our CDN.
Jul 8, 19:45 UTC
Update -
Issues and Pages are operating normally.
Jul 8, 19:44 UTC
Jul 8, 19:44 UTC
Update -
Issues and Pages are experiencing degraded performance. We are continuing to investigate.
Jul 8, 19:44 UTC
Jul 8, 19:44 UTC
Update -
Issues is operating normally.
Jul 8, 19:44 UTC
Jul 8, 19:44 UTC
Update -
Issues is experiencing degraded performance. We are continuing to investigate.
Jul 8, 19:44 UTC
Jul 8, 19:44 UTC
Update -
Pages and Issues are operating normally.
Jul 8, 19:44 UTC
Jul 8, 19:44 UTC
Update -
Our assets are serving normally again and all impact is resolved.
Jul 8, 19:44 UTC
Jul 8, 19:44 UTC
Update -
We are beginning to see recovery of our assets and are monitoring for additional impact.
Jul 8, 19:16 UTC
Jul 8, 19:16 UTC
Update -
githubstatus.com may not be available or may be degraded for some users in some regions.
Jul 8, 19:12 UTC
Jul 8, 19:12 UTC
Update -
We are investigating issues with loading assets, including JavaScript assets, on various parts of the site for some users.
Jul 8, 19:02 UTC
Jul 8, 19:02 UTC
Investigating -
We are investigating reports of degraded performance for Issues and Pages
Jul 8, 19:01 UTC
Jul 8, 19:01 UTC
Jul 7, 2024
No incidents reported.
Jul 6, 2024
No incidents reported.
Jul 5, 2024
Resolved -
On July 5, 2024, between 16:31 UTC and 18:08 UTC, the Webhooks service was degraded, with customer impact of delays to all webhook delivery. On average, delivery delays were 24 minutes, with a maximum of 71 minutes. This was caused by a configuration change to the Webhooks service, which led to unauthenticated requests sent to the background job cluster. The configuration error was repaired and re-deploying the service solved the issue. However, this created a thundering herd effect which overloaded the background job queue cluster which put its API layer at max capacity, resulting in timeouts for other job clients, which presented as increased latency for API calls.
Shortly after resolving the authentication misconfiguration, we had a separate issue in the background job processing service where health probes were failing, leading to reduced capacity in the background job API layer which magnified the effects of the thundering herd. From 18:21 UTC to 21:14 UTC, Actions runs on PRs experienced approximately 2 minutes delay and maximum of 12 minutes delay. A deployment of the background job processing service remediated the issue.
To reduce our time to detection, we have streamlined our dashboards and added alerting for this specific runtime behavior. Additionally, we are working to reduce the blast radius of background job incidents through better workload isolation.
Jul 5, 20:57 UTC
Shortly after resolving the authentication misconfiguration, we had a separate issue in the background job processing service where health probes were failing, leading to reduced capacity in the background job API layer which magnified the effects of the thundering herd. From 18:21 UTC to 21:14 UTC, Actions runs on PRs experienced approximately 2 minutes delay and maximum of 12 minutes delay. A deployment of the background job processing service remediated the issue.
To reduce our time to detection, we have streamlined our dashboards and added alerting for this specific runtime behavior. Additionally, we are working to reduce the blast radius of background job incidents through better workload isolation.
Jul 5, 20:57 UTC
Update -
We are seeing recovery in Actions start times and are observing for any further impact.
Jul 5, 20:44 UTC
Jul 5, 20:44 UTC
Update -
We are still seeing about 5% of Actions runs taking longer than 5 minutes to start. We are scaling and shifting resources to encourage recovery of the problem.
Jul 5, 20:32 UTC
Jul 5, 20:32 UTC
Update -
We are still seeing about 5% of Actions runs taking longer than 5 minutes to start. We are evaluating mitigations to increase capacity to decrease latency.
Jul 5, 19:58 UTC
Jul 5, 19:58 UTC
Update -
We are seeing about 5% of Actions runs not starting within 5 minutes. We are continuing investigation.
Jul 5, 19:19 UTC
Jul 5, 19:19 UTC
Update -
We have seen recovery of Actions run delays. Keeping the incident open to monitor for full recovery.
Jul 5, 18:40 UTC
Jul 5, 18:40 UTC
Update -
Webhooks is operating normally.
Jul 5, 18:10 UTC
Jul 5, 18:10 UTC
Update -
We are seeing delays in Actions runs due to the recovery with webhook deliveries. We expect this to resolve with the recovery of webhooks.
Jul 5, 18:09 UTC
Jul 5, 18:09 UTC
Update -
Actions is experiencing degraded performance. We are continuing to investigate.
Jul 5, 18:07 UTC
Jul 5, 18:07 UTC
Update -
We are seeing recovery as webhooks are being delivered again. We are burning down our queue of events. No events have been lost. New webhook deliveries will be delayed while this process recovers.
Jul 5, 17:57 UTC
Jul 5, 17:57 UTC
Update -
Webhooks is experiencing degraded performance. We are continuing to investigate.
Jul 5, 17:55 UTC
Jul 5, 17:55 UTC
Update -
We are reverting a configuration change that is suspected to contribute to the problem with webhook deliveries.
Jul 5, 17:42 UTC
Jul 5, 17:42 UTC
Update -
Our telemetry shows that most webhooks are failing to be delivered. We are queueing all undelivered webhooks and are working to remediate the problem.
Jul 5, 17:20 UTC
Jul 5, 17:20 UTC
Update -
Webhooks is experiencing degraded availability. We are continuing to investigate.
Jul 5, 17:17 UTC
Jul 5, 17:17 UTC
Investigating -
We are investigating reports of degraded performance for Webhooks
Jul 5, 17:04 UTC
Jul 5, 17:04 UTC
Jul 4, 2024
No incidents reported.
Jul 3, 2024
Resolved -
On July 3, 2024, between 1:34 PM UTC and 4:42 PM UTC the GitHub documentation was degraded and showed a 500 on non-cached pages. On average, the error rate was 2-5% and peaked at 5% of requests to the service. This was due to an observability misconfiguration. We mitigated the incident by updating the observability configuration and redeploying. We are working to reduce our time to detection and mitigation of issues like this one in the future.
Jul 3, 16:40 UTC
Jul 3, 16:40 UTC
Update -
Mitigation measures have been rolled out and we're seeing errors disappear in our telemetry. We'll continue to monitor our services closely to ensure the docs site is fully healthy.
Jul 3, 16:37 UTC
Jul 3, 16:37 UTC
Update -
We have identified a likely cause of the errors with GitHub Docs and are working on a mitigation.
Jul 3, 15:59 UTC
Jul 3, 15:59 UTC
Investigating -
We are currently investigating this issue.
Jul 3, 15:24 UTC
Jul 3, 15:24 UTC
