Loss of A+B power to one cabinet and loss of A feed to remaining cabinets at Volta Data Centres
Incident Report for Clouvider
Resolved
All servers should now be back online; If you are facing any issues with your service, please reach out to support for assistance.

No official update from site, as such, the site should continue to be classified as at risk. We will provide additional updates once the site confirms resolution.

Taking into account, among others, recent issues at the site Clouvider has planned to move out from this facility and re-locate the services to our new cage in another building. In order to ensure continued high quality of service, these plans will now be accelerated and, if your service has been affected, you will hear from us separately, via the ticket system once the migration of your service has been scheduled.

We apologise for the inconvenience caused.
Posted Feb 03, 2024 - 13:55 GMT
Monitoring
All feeds are now back online in all cabinets.
Network equipment in the affected cabinet is back online and some of the servers are now also back online.

Our team is ensuring all active servers are powered up.

No official update from site, as such, the site should continue to be classified as at risk.
Posted Feb 03, 2024 - 13:49 GMT
Update
As per the site:

"We have identified the source of a control failure that was holding open circuit breakers feeding critical load on two of our systems.

Customers that are fed from both of these systems will have been impacted by the maintenance event.

We expect restoration of critical power within the next 90 minutes. We will then work to ensure that all cabinets that have been impacted are back online."
Posted Feb 03, 2024 - 13:31 GMT
Update
After further escalation, the site has sent out an update confirming they are continuing to investigate and advising that during their planned works on the generator switchboard system of the A-feed, that were not expected to be service affecting (and most certainly not to both A and B feeds simultaneously), they suffered unexpected circuit breaker trips that in turn affected power to some of their Customers.

Our engineer is on site ready to reinstate the services. As for the cabinets with both A and B feeds affected, another engineer is en-route with additional equipment, including PDU extension cables that might allow us to bring the cabinet back online using power from a nearby empty cabinet.

With regards to the remaining cabinets, B-feed seems to be holding stable. A-feed continues to be unavailable to all of our cabinets at this site.
Posted Feb 03, 2024 - 12:51 GMT
Update
We now lost one power feed to remaining cabinets. Currently only B feed is operational in remaining cabinets with A+B feed down in one cabinet.

At this point the site should be considered at risk of an outage affecting all services in the building.

We are awaiting additional updates from the site.
Posted Feb 03, 2024 - 11:54 GMT
Identified
The site has confirmed power issues and that they are working on them right now.
We don't currently understand how they come to affect both A and redundant B feeds simultaneously to this cabinet.

Our engineer is ready to reinstate the services as soon as the power is back online.
Posted Feb 03, 2024 - 11:40 GMT
Investigating
We are observing loss of connectivity to multiple devices in one of the cabinets at Volta Data Centres. It appears to be power related.

The site has been contacted and our engineer is on the way to the site to investigate.
Posted Feb 03, 2024 - 11:24 GMT
This incident affected: Network (London, UK Core Network), Datacentres (Volta - London, UK), and Hosted Services (Dedicated Servers).