Research reportData-center capacity

What causes data-center outages: surveys and incident reviews

Uptime's 2025 survey puts power behind 45% of impactful outages. Published incident reviews show how faults in power, cooling and water spread, and what set recovery time.

Cover of the full report: What causes data-center outages: surveys and incident reviews
The research behind this article, with every source listed. 22 pages.

Key points

From the Summary of the full report

  1. Half of the operators in Uptime Institute's 2025 survey reported at least one impactful outage, one significant enough to record, at their facility in the past three years, down from 74% in 2020 [1]. One in 10 said their most recent one was serious or severe [1].
  2. Power caused 45% of the most recent impactful outages in Uptime Institute's 2025 survey, down from 54% in 2024, with cooling at 14% and network at 11% [1]. Uptime's 2026 survey puts power and cooling together at more than two-thirds [5].
  3. Uptime's Data Center Resiliency Survey 2025 found uninterruptible power supply (UPS) failures behind 42% of power-related IT service outages, transfer switch failures behind 36% and generator failures behind 28% [1].
  4. In Uptime's pooled 2025 and 2026 survey responses, 70% of respondents whose cooling had no spare capacity (N) reported an impactful outage in three years, against 39% of those with fully duplicated cooling (2N), a correlation Uptime's authors caution is not causation [5].

Data center outage causes show up in two kinds of public record. Uptime Institute's annual survey asks operators what caused their most recent impactful outage, and in 2025 power led with 45%, ahead of cooling at 14% and network at 11% [1]. Incident reviews published by cloud providers, a grid reliability body and a regulator show what a single share cannot: how one fault got past the protection built to contain it, and what decided how long recovery took [15, 17, 18, 21, 24].

Both records have gaps. Uptime says outages "are often under-reported or disclosed with limited details" [1], and each review is its author's own account.

Data center outage causes in Uptime's survey

Half of the operators in Uptime's 2025 survey reported at least one impactful outage, one significant enough to record, in the past three years, down from 74% in 2020 [1]. One in 10 rated their most recent one serious or severe [1]. Among those whose outage was significant, 20% put its cost above $1 million [1].

Primary cause, 2025 Share of most recent impactful outages
Power 45%
Cooling 14%
Network 11%
IT systems 9%
Fire suppression 6%
Information security-related 5%

Source: [1], 96 answers, rounded.

The power share fell from 54% in 2024 [1]. Uptime's 2026 survey puts power and cooling together at more than two-thirds of impactful outages [5]. In June 2023, Uptime put the frequency of power-related outages partly down to "power system complexity and a dependence on moving parts", and noted that compared with other systems such as cooling, "power is more prone to fail completely, rather than operating at partial capacity" [6].

Inside power failures: UPS, transfer switches and generators

Uptime's Data Center Resiliency Survey 2025 found uninterruptible power supply (UPS) failures behind 42% of power-related IT service outages, transfer switch failures behind 36% and generator failures behind 28% [1]. In Uptime's Resiliency Survey 2023, more than 70% of organizations ran pull-the-plug tests, and less than half of operators involved their utility [7]. Uptime warns that without a full disconnection, "lurking failures in transfer switches and paralleling switchgears may go undetected" [6].

Two reviews show the chain. During a June 2012 storm in Northern Virginia, a voltage spike started a transfer to generator power in two AWS data centers, and in one of them "each generator independently failed to provide stable voltage" [16]. The UPS systems ran down and servers lost power, although the generators were less than two years old and tested weekly [16]. In November 2023, at a facility Cloudflare leases near Hillsboro, Oregon, Cloudflare's post-mortem says the protective response to a transformer ground fault also shut down the generators, and that batteries meant to last about 10 minutes "started to fail after only 4 minutes" [20].

Does redundancy prevent outages?

Uptime's October 2026 analysis pooled over 1,200 survey responses [5]. Of respondents with N cooling, capacity sized to the expected peak load with no spare, 70% reported an impactful outage in three years, against 39% with 2N, where the full capacity is duplicated [5]. For power the figures were 68% and 42% [5]. The authors write that "correlation is not causation" and that the N samples are small [5]. The explainer on data-center Tier levels sets out how the Tiers build on these terms.

Redundant equipment can share one trigger. A control system bug in AWS's Tokyo region in 2019 "caused multiple, redundant cooling systems to fail" [15]. Google reported "a simultaneous failure of multiple, redundant cooling systems" in its europe-west2-a zone in July 2022, in extremely high outside temperatures [19].

When cooling controls and chillers fail

In Tokyo, the cooling was designed to run at maximum if its control system failed. In part of the data center it shut down instead, purge failed too, and AWS traced both to the programmable logic controllers [15]. With "minimum visibility", the team reset each unit by hand [15].

In Microsoft Azure's Australia East region on 30 August 2023, a lightning strike about 18 miles away caused a voltage sag that stopped chillers for multiple data centers, and 13 did not restart on their own [17]. Engineers restarted them one data center after another, and by the last five the chilled water was too hot to allow it [17]. Microsoft changed to a load-based restart order that gives every hall partial cooling, and wrote that staffing at the time "would have been sufficient to prevent impact" with that order [17].

CME Group's annual report says a "critical cooling failure caused by human error" at its data center operated by CyrusOne led it to halt its markets on 27 November 2025 [26].

Water, batteries and fire

In April 2023, a cooling water pipe leak in Google Cloud's europe-west9 region entered a UPS room and led to a fire [18]. The building's fire suppression water ran out, the fire marshal ordered one building shut down, and the fire took nearly 16 hours to put out [18].

In Pangyo in October 2022, according to police reports Uptime cited, a fire at a colocation facility "started in a battery room before spreading quickly to the rest of the building" [12]. In Daejeon in September 2025, Korean press reported a lithium-ion battery fire during a relocation, and in September 2026 an audit was reported to have found only one of eight battery racks powered down [28, 30]. Uptime wrote that lithium-ion batteries carry a greater fire risk than valve-regulated lead-acid ones, and that fire-resistant battery compartments can reduce the risk of facility-wide outages [12].

Grid faults and transfers to backup power

On 10 July 2024, a lightning arrestor failed on a 230 kV line in the Eastern Interconnection, and the line's automatic reclosing produced six faults in 82 seconds [21]. About 1,500 MW of "exclusively data center-type load" left the grid as data centers transferred to backup power, which grid operators had not anticipated [21]. NERC's review does not report any service outage at the data centers [21]. About 1,260 MW did not return for hours, and NERC describes protection schemes that keep the load off the grid until staff transfer it back by hand [21].

What set recovery time

Recovery depended on what came after the fault.

  • Google's regional Spanner had two of its three replicas in the building that was powered down, which interrupted control-plane services across europe-west9 [18].
  • Australia East reached full mitigation on 3 September, about four days and 11 hours after the sag (derived from [17]), held back by storage and database dependencies [17].
  • In Singapore in October 2023, a cooling malfunction at the data center hosting DBS and Citibank systems shut them down, and the banks could not fully recover at their backup data centers, DBS because of a network misconfiguration and Citibank because of connectivity issues [24]. Annual recovery exercises had not surfaced either fault [24].

What capacity contracts say about outages

Digital Realty reports that "substantially all" of its customer agreements carry service level commitments, and that a failure could bring credits against rent and, in certain cases of repeated failures, a customer's right to terminate [31]. Equinix says that in some leased buildings the landlord is responsible for the UPSs and generators [32]. CoreWeave, a GPU cloud that leases its space, says it does not control the operation of those facilities and has provided service credits or refunds [33]. None of the filings gives credit percentages [31, 32, 33]. The research on financeable GPU capacity contracts covers how service credits carry over to a lender.

Questions to settle before signing a capacity agreement

Rillor's data-center capacity service sources and contracts power and space for GPU systems in blocks from 250 kW to 1 MW, each matched to the systems it runs by power draw and cooling grade. The reviews above suggest these questions for any site:

  • How the transfer to generator is tested, whether the utility takes part, and the last result [6, 7].
  • What UPS design and battery chemistry the hall uses, and whether battery rooms are compartmented [12].
  • What the cooling controls do when they fail [15].
  • Whether chillers restart on their own after a voltage sag, in what order, and who restarts them by hand [17].
  • Whether water pipes run above or beside UPS and battery rooms [18].
  • The hall's power and cooling redundancy, in N terms [5].

In November 2025, Uptime reported that in the worst cases GB200 NVL72 racks can see sudden power climbs from around 60 to 70 kW to more than 150 kW, above their 132 kW maximum specification, and that engine generators would be the most stressed power equipment if exposed to those swings [10]. The rack power explainer covers what those racks draw.

Read the full report

The full report adds the outage frequency table by year, the full cause table, incident timelines, the outage reductions for N+1 and N+2, and the limits of each source.

The full report

What causes data-center outages: surveys and incident reviews

The research behind this article, with every source listed. 22 pages.

  1. Summary
  2. Background
  3. Method
  4. Findings
  5. Implications
  6. Limits of this analysis
  7. Sources

Sources

  1. Uptime Institute Global Data Center Survey 2025. Douglas Donnellan, Andy Lawrence, Daniel Bizo, Owen Rogers and others, Uptime Institute, July 2025, viewed 8 November 2026. https://datacenter.uptimeinstitute.com/rs/711-RIA-145/images/2025.Annual.Survey.Report.pdf
  2. Survey findings link higher redundancy to lower outage risk. Owen Rogers and Douglas Donnellan, Uptime Institute Journal, 14 October 2026, viewed 8 November 2026. https://journal.uptimeinstitute.com/survey-findings-link-higher-redundancy-to-lower-outage-risk/
  3. Data center operators will face more grid disturbances. Jacqueline Davis, Lenny Simon, Daniel Bizo and Max Smolaks, Uptime Institute Journal, 28 June 2023, viewed 8 November 2026. https://journal.uptimeinstitute.com/data-center-operators-will-face-more-grid-disturbances/
  4. Are utility companies needed for pull-the-plug testing? Douglas Donnellan, Uptime Institute Journal, 20 September 2023, viewed 8 November 2026. https://journal.uptimeinstitute.com/are-utility-companies-needed-for-pull-the-plug-testing/
  5. Electrical considerations with large AI compute. Daniel Bizo, Uptime Institute Journal, 19 November 2025, viewed 8 November 2026. https://journal.uptimeinstitute.com/electrical-considerations-with-large-ai-compute/
  6. Major data center fire highlights criticality of IT services. Max Smolaks and Daniel Bizo, Uptime Institute Journal, 11 January 2023, viewed 8 November 2026. https://journal.uptimeinstitute.com/major-data-center-fire-highlights-criticality-of-it-services/
  7. Summary of the Amazon EC2 Issues in the Asia Pacific (Tokyo) Region (AP-NORTHEAST-1). Amazon Web Services, 25 August 2019, updated 28 August 2019, viewed 8 November 2026. https://aws.amazon.com/message/56489/
  8. Summary of the AWS Service Event in the US East Region. Amazon Web Services, 2 July 2012, viewed 8 November 2026. https://aws.amazon.com/message/67457/
  9. Post Incident Review (PIR): Services impacted after power/cooling issue, Australia East (Tracking ID VVTQ-J98). Microsoft Azure status history, event of 30 August 2023, viewed 8 November 2026. https://azure.status.microsoft/en-us/status/history/?trackingId=VVTQ-J98
  10. Multiple Google Cloud services in the europe-west9-a zone are impacted (incident report). Google Cloud Service Health, full report 23 June 2023, viewed 8 November 2026. https://status.cloud.google.com/incidents/dS9ps52MUnxQfyDGPfkY
  11. Multiple Cloud products experiencing elevated error rates, latencies or service unavailability in europe-west2 (incident report). Google Cloud Service Health, full report 29 July 2022, viewed 8 November 2026. https://status.cloud.google.com/incidents/fmEL9i2fArADKawkZAa2
  12. Post mortem on the Cloudflare Control Plane and Analytics Outage. Matthew Prince, Cloudflare blog, 4 November 2023, viewed 8 November 2026. https://blog.cloudflare.com/post-mortem-on-cloudflare-control-plane-and-analytics-outage/
  13. Incident Review: Considering Simultaneous Voltage-Sensitive Load Reductions. North American Electric Reliability Corporation, 8 January 2025, viewed 8 November 2026. https://www.nerc.com/globalassets/our-work/reports/event-reports/incident_review_large_load_loss.pdf
  14. Oral reply to Parliamentary Questions on banking services disruption of DBS and Citibank on 14 October 2023. Monetary Authority of Singapore, 6 November 2023, viewed 8 November 2026. https://www.mas.gov.sg/news/parliamentary-replies/2023/oral-reply-to-parliamentary-question-onbanking-services-disruption-of-dbs-and-citibank
  15. CME Group Inc. Annual Report on Form 10-K for 2025. U.S. Securities and Exchange Commission, filed 26 February 2026, viewed 8 November 2026. https://www.sec.gov/Archives/edgar/data/1156375/000115637526000009/cme-20251231.htm
  16. Government struggles to restore services after data center blaze. The Korea Herald, 28 September 2025, viewed 8 November 2026. https://www.koreaherald.com/article/10585032
  17. Board of Audit and Inspection: "National Information Resources Service Fire Was Man-Made... Unregistered Contractor Used for Illegal Subcontracting". The Asia Business Daily, 7 September 2026, viewed 8 November 2026. https://view.asiae.co.kr/en/article/2026090710145353550
  18. Digital Realty Trust, Inc. Annual Report on Form 10-K for 2025. U.S. Securities and Exchange Commission, filed 13 February 2026, viewed 8 November 2026. https://www.sec.gov/Archives/edgar/data/1297996/000110465926015365/dlr-20251231x10k.htm
  19. Equinix, Inc. Annual Report on Form 10-K for 2025. U.S. Securities and Exchange Commission, filed 11 February 2026, viewed 8 November 2026. https://www.sec.gov/Archives/edgar/data/1101239/000110123926000032/eqix-20251231.htm
  20. CoreWeave, Inc. Annual Report on Form 10-K for 2025. U.S. Securities and Exchange Commission, filed 2 March 2026, viewed 8 November 2026. https://www.sec.gov/Archives/edgar/data/1769628/000176962826000104/crwv-20251231.htm

Notices

Rillor is not a registered investment adviser or commodity trading advisor and does not provide investment or trading advice. Research, forecasts, data and software described on this site are for research and engineering use. Nothing here is an offer or recommendation to buy or sell any security, commodity interest or digital asset. Past or simulated results do not indicate future results.

Reserve power before the hardware arrives.

Send the GPU systems you plan to run, the block size and cooling you need, and the month you need power.

Discuss capacity