A Disaster Recovery Perfect Storm

Last week I had a customer with a downed POWER5 system due to hardware failure.


The situation:

The system bezel said that it couldn’t find the disk drives which could’ve been a number of failing parts. This system had a single disk drive mirrored to another. The backup was questionable (i.e., the customer couldn’t guarantee what was on it) on ¼ inch tapes so it was not really compatible with current technology. Also, the customer did not have an active hardware/software maintenance contract. The system was set up for operations console but there was no cable nor operations console PC in the building. No LIC DVD was available either. These are the facts of the situation.


Settling on a plan:

An IBM customer engineer (CE) was dispatched at their hourly rate with a four-hour minimum and a purchase order required up front. That’s standard. IBM arrived on-site at 2 PM as they were a 90-minute drive away. Since there was no available console and no LIC DVD present, plus the CE didn’t bring a 5.4 LIC DVD, there’s not much they could do that late in the day. When you’re paying IBM time and materials, they don’t necessarily work on the weekend. Time and materials mean when they have the time and if they have the materials, within a 9 to 5 window during the business day. Customers with 24×7 maintenance contracts come first so we contemplated the scenarios with IBM about how to move forward.


We settled on a two-prong approach: we would prepare a cloud recovery and IBM will continue diagnosing the problem locally.