Guido Vrola - Fotolia
Survey: Many businesses neglecting disaster recovery testing
Even though DR testing is a low-cost way to ensure readiness, businesses don't do it enough. Roadblocks include limited time, stretched IT teams and basic human nature.
How quickly can you get your business running again if your data center is hit by a flood or fire?
That is a question every business needs to ask.
At least one analyst said most businesses think they are ready for natural disasters, despite not testing their disaster recovery (DR) plans regularly. A recent survey centered around ransomware readiness arrived at a similar conclusion -- many businesses are confident in their ability to recover from attacks, yet most don't perform disaster recovery testing more than once a year.
Marc Staimer, president of Dragon Slayer Consulting, said disaster recovery testing is one of the lowest-cost ways for a business to understand and potentially increase their preparedness for disasters. He said most vendors that offer DR as a service will do DR testing. However, most of the businesses he encounters skip testing altogether and often just hope the plan they have in place will work in a crisis.
"It's human wiring. Culturally, it's harder to fix," Staimer said.
Staimer said that from a business perspective, disaster recovery testing does not generate revenue and consumes time and resources. But from the psychological side, there's the notion that nothing bad will ever happen, or that it will never be severe enough to warrant a bulletproof plan. If an organization had to make a cut somewhere, DR is a likely target.
Staimer said there is little that vendors, analysts or IT publications can do to change this kind of thinking, because this isn't a matter of lack of information or awareness. However, for better or worse, the increasing frequency of natural disasters is causing more businesses to evaluate their DR plans.
"Anxiety is a marvelous motivator, so it's moving the needle a little," Staimer said.
Infrequent disaster recovery testing
A study commissioned by backup vendor StorageCraft and performed by Dimensional Research asked about 700 companies in Australia, North America, Germany, France and the U.K. about their recoverability after a ransomware attack. The report, published in December 2019, found 68% of respondents were confident in their ability to recover quickly, but 22% said they do not test their DR plan at all. Of those that do, 37% said they test annually, while only 18% do so monthly.
Shridar Subramanian, vice president of marketing and product management at StorageCraft, said there is a disconnect between perception and reality when it comes to businesses' DR readiness. He argued the confidence expressed by the respondents does not match up with the level of testing they do to ensure their recoverability.
"Attacks are evolving constantly, but people are only testing their defenses once a year," Subramanian said.
The survey found money to be the main roadblock to adequate recovery following a failure. 46% of respondents said they have the skills but not the budget to recover from failure, and another 9% said they lack both the skills and the budget.
Subramanian said doesn't see the lack of DR testing as a problem of hubris, but one of IT being stretched thin. He said more pressing problems force administrators' attention and resources away from DR. Simple DR testing falls by the wayside when administrators are tasked to deal with rapid data growth and increasingly complex and siloed environments.
A stroke of luck
Professional rugby league club Leeds Rhinos, based out of Leeds, West Yorkshire, England, was hit by the Boxing Day floods of 2015. Because it hit on a holiday and the flood came so quickly, the water in the club's Kirkstall Training Ground building was chest-high before the club could react. The team had its servers, training videos and analytical data in that building.
The Leeds Rhinos do not have in-house IT staff, and the team contracts with managed service provider MTech IT Solutions, also based in Leeds. MTech is responsible for the club's backup and DR at all of its sites, which include Kirkstall Training Ground and the primary servers at Headingley Rugby Stadium.
Stephen Green, technical project manager at MTech, said it could have been worse. While the training facility held important data, it was not the primary data center for the Rhinos. The club had a backup system that took hourly snapshots and sent them off to the primary site. Because the disaster occurred over Christmas, Kirkstall Training Ground was closed until the rugby season began again in February.
"It took a little bit of the urgency off of it," Green said. "I think it was very lucky that it was a flood down at the training ground and not something else at the primary site."
The Kirkstall Training Ground servers were restored between two and three days, which Green admitted was slow from a business continuity perspective. Had there been a more pressing need to get those servers back online, he would've had to drop in an emergency DR server.
A disaster at Headingley Stadium would've been a different story. Green said the main site consists of 15 to 20 servers. These servers held the club's most business-critical data, including financial records and employee data.
"If the disaster had been there, we would've been in a whole world of hurt, and we would've had to get things up again a lot faster," Green said.
Since BC/DR was more critical at the stadium, MTech implemented StorageCraft's HeadStart Restore for pre-staging recoveries, lowering downtime. After the flooding at the training ground, MTech has put a similar setup in place for that location as well.
The office at Kirkstall Training Ground was badly damaged by the flood and had to be completely renovated. It did not open again until July 2016.
