Friday, May 22, 2009

Business Continuity Series, pt 5 - Implementing the Plan

Once all the homework has been done understanding the relationships and inter-dependencies and once the plan has been put together, it's time to test and implement the plan.

At this stage there is a tricky conundrum. On one hand, in order to do a realistic test, planned outages are required for live services to ensure that systems will be resilient in the manner anticipated (reducing the shock of discovering additional failures during an actual disaster). However, deliberately causing outages is the last resort of any service provider.

Even in the best conditions, where service consumers are notified in advance with a long lead times and everything goes according to plan it is usually heavily orchestrated event that consumes many non-revenue generating resources.

Testing the plan should attempt to avoid being disruptive. As with any change management procedure, downtime should be kept to a minimum and attempts should be made to reduce the impact on live customers (usually translating into "off-peak testing", coming in late on a Saturday night or early Sunday morning).

The shutdown of highly technical and regulated services like nuclear power plants usually requires all hands on deck at the most ungodly hours of the night (colleagues of mine working with in nuclear power remind me that their credo is "Never forget that you work in a very unforgiving industry").

In this stage, often managers and professionals discover more inter-dependencies implying that their plans are either incomplete or not as robust as they had anticipated. This is where GAP analysis comes into play to further develop the plans.

Even in the event of an ideal and perfect implementation of a BCP plan, there is still the requirement of ongoing vigilance. This is because that as the environment changes, certain assumptions which become obsolete suddenly cause vulnerabilities to appear in the system. At this point, BCP projects evolve into on-going BCP maintenance programs.

No comments: