The tech giants were forced to repair servers as cooling systems struggled with outside temperatures of 40C
To live
Image: Stockfresh
Both Google and Oracle cloud servers were hit by yesterday’s intense heat as data centers in south London were hit by a series of problems caused by a cooling failure.
In a service status update Google referred to IT Pro, the company identified 9:15 AM PDT as the point where multiple services started experiencing issues:
“Multiple cloud products with increased error rates, latencies or service unavailability in europe west2.”
advertisement
“We are experiencing an issue with Google Cloud Storage, Google BigQuery, Google App Engine, Google Cloud Functions, Google Cloud Dataflow,” the company wrote in a detailed description of the issues.
As of 7:06 PM PDT, Google reported that the affected cooling systems had been restored. Oracle experienced similar cooling system failures and explained that this was the cause of delays in accessing customers with Oracle Cloud Infrastructure.
Such a customer with assets hosted in the UK South (London) region may have encountered issues. Oracle was asked for a statement and pointed IT Pro to the latest status Reportacknowledging that the issue is “now resolved”.
“After unusually high temperatures in the UK South (London) region, two cooler units in the data center experienced a failure when they were forced to operate beyond their design limits,” read the executive summary.
“As a result, temperatures in the data center started to rise, shutting down some of the compute infrastructure.”
Temperatures of over 40 degrees Celsius were experienced in parts of London on Tuesday, marking the UK’s hottest day on record. Normal business operations were suspended for many employees, and some companies told employees to work from home.
Meteorologists had warned for days ahead of the weather that critical infrastructure could be compromised as the UK faced unprecedented temperatures.
Luton Airport reported that a small section of the runway had been lifted by the heat, and train services were disrupted on a large scale while the tracks bent and the overhead wires melted. In some parts of the UK, firefighters had to tackle forest fires.
Both Google and Oracle identified portions of the affected systems that were still experiencing issues after their respective incidents ended. Google specified that a “small number of HDD-backed Persistent Disk volumes will still be impacted and will show IO errors.”
Oracle identified the systems that were later repaired and the systems that had not:
“Instances of virtual machines that failed to self-repair were manually put online by service technicians. However, a subset of Oracle Integration Cloud resources continue to be impacted. Engineers are actively working to reduce the remaining service resources.”
© Dennis Publishing
