[OIT-Colocation] False fire alarms and cooling emergency yesterday

[Allen Schiano]

Colo partners in the OIT data center

We had an emergency temp condition yesterday in the data center between 12:30 and about 1:15.  A faulty fire alarm sensor set off the fire alarms in the data center.  By safety code that causes the fans in the HVAC units to shut down so as not to cause a fire to spread.

But with no fire we overheated the room to over 100F for a few minutes.  As such, your computers could gave shut down.

So far we have received only one report of possible damage.  But we have contacted Risk Management to start the insurance process.

Meanwhile EH&S has assured us that the fire alarm system is stable and the faulty sensor replaced.  We are meeting with them and FM soon to discuss the event in detail.

The response to automated warnings was prompt by OIT and other staff so we limited to the time of the event.  We are working with FM and EH&S to get even more timely warnings so that we could minimize this type of event from overheating the room as much as this time.  But it does show quite vividly how 300KW of computer heat can do to a 7000 sq ft data center!

Again, sorry for the event occurring.  Please keep me informed of any damage your equipment might have sustained.

Allen

Sent from my iPhone