The Strangest Data Center Outages

Brian Wood Blog

A power outage represents a major emergency not just for a data center, but the countless clients and customers that depend on it for a multitude of business and personal needs. Downtime means unhappy people and ultimately, lost dollars. Of course, each of these centers has auxiliary power supplies, but sometimes, even they lose juice. Typically, a power outage can be attributed to an easily-identifiable source.

However, there are other times when there is seemingly no explanation for the outage – or the reason seems so random or improbable that you think it has to be made-up.

Here are a few of the strangest data center outages that have occurred:

Anchors Aweigh:

Undersea cables allow data to be shared from place to place and from continent to continent. Most ships are completely unaware of this fact and have been known to drop their anchors right on the top of them. 2008 was a particularly bad year for this, especially in the Mediterranean Sea, where numerous instances and outages were noted (colocationamerica.com).

outage 1

A ship’s anchor (cruelkev2.blogspot.com)

Squirrels vs. Yahoo:

Yes, squirrels chew everything and yes, one brave squirrel sacrificed himself for all rodent-kind in 2010 when he chewed through an important communication transferring wire, disabling about half of the company’s Santa Clara data center (colocationamerica.com). The squirrel was dubbed the “frying squirrel” by Yahoo employees.

The Perfect Storm of Sandy:

Data centers are more than prepared to handle the absolute worst that Mother Nature can throw at them. However, when Mother Nature throws everything at them at once (including the kitchen sink, literally), there isn’t much that preparedness can overcome. Torrential downpours, high tide, a full moon, and a major deviation of the high-altitude jet stream helped Hurricane Sandy storm take aim on the east coast. Basically, everything came together at the same time to make this type of outage; a truly rare climatic occurrence.  “Sandy” became known as a Superstorm.

outage 3

Superstorm “Sandy” (mesa-arc.org)

Migration Highway:

Moving servers can be a tricky business. NaviSite (now owned by  Time Warner), acquired a hosting provider called Alabanza in 2007 and was moving customer accounts from Alabanza’s main data center in Baltimore to a facility in Andover, Massachusetts. They literally unplugged the servers, put them on a truck, and drove them over 400 miles. Many websites hosted by Alabanza were reportedly offline for the duration of the drive and the re-installation (datacenterknowledge.com).

(A similar move-related problem occurred a few months earlier, when Hostway moved ValueWeb servers from Miami to Tampa. Hostway said later that more than 500 servers suffered hardware failures when they were restarted in the new facility).

outage 4

U-Haul moving truck (uhaul.com)

When One Door Closes:

Nianet, a Danish ISP, went down when thieves cut a hole in its Taastrup data center’s walls to sneak in and steal components. They walked off with a bunch of networking cards, according to news reports. How the thieves were able to cut through a data center wall, and why they did so just to steal networking cards, remains a mystery (datacenterknowledge.com).

outage 5

Thieves cut a hole into wall (itv.com)

The Leap-Second Bug:

A “leap second” is when people in charge of the Universal Time adjust that time by a second to accommodate minute changes in the Earth’s rotational speed. In 2012, the Universal Time was adjusted and it caused a bug which downed several popular sites like LinkedIn, Reddit, Mozilla, and The Pirate Bay while also causing 400 Qantas flights in Australia to be delayed as they had to switch to manual check-ins.

outage 6

Clock showing universal time (3news.co.nz)

Keep Your Butts To Yourself:

At least one data center downtime incident was sparked by smoldering mulch. The Perth iX data center in Western Australia shut down for an hour after its VESDA (Very Early Smoke Detection Apparatus) system detected smoke in the data center. The cause was identified to be a smoldering mulch-filled garden bed alongside the outside wall of the facility, most likely lit by a burning cigarette butt (datacenterknowledge.com). Let that be a lesson: cigarette butts are responsible for much more than simply forest fires.

outage 7

Cigarette butt slowly starting a fire (1310news.com)

Sources:
datacenterknowledge.com
colocationamerica.com