Rob Byron - Fotolia
How to recover from a data backup failure and move forward
Backup failures don't need to cause a long-term crisis for your organization. Make sure you're ready for this unfortunate possibility with these practical best practices.
While believing "failure isn't an option" is an admirable outlook to have, a backup failure can still find a way to raise its ugly head -- and often at the most inopportune times.
When data goes missing, and its backup is either absent or defective, a genuine crisis has arrived. Successfully resolving the situation and preventing a complete catastrophe requires careful thought, analysis and patience.
First, don't panic -- there will be plenty of time for that later. Also, don't focus on why the backup failed, advised David Schwed, professor and director of the cybersecurity program at Yeshiva University. "There's time for a post-mortem after you exhaust all possible remedies."
The road to recovery
It's important to determine the exact date of the backup failure and attempt recovery with the last good backup. "Then, identify what data is missing between the last known successful backup and the failed backup," Schwed said.
The next step is to look for alternative ways of reconstructing any important missing data. A retailer, for example, may be able to reconstruct missing transaction data with some help from its credit card processor. Even email or paper receipts, if available, can be combed to rebuild critical sales data.
"You may not recover all of the missing data, but at least you will have identified what's lost and, hopefully, can reconstruct some of that missing data," Schwed said.
Determining the origin point of a backup failure is also a key step in recovering from a cloud service data backup collapse.
"Look through the log files to determine a time period," said Doug Hazelman, vice president of technical marketing for CloudBerry Lab, a cloud backup service provider based in Lewes, Del.
If it's not obvious when the failure occurred, collect all the log files you can and open a ticket with your backup vendor.
"Supply them the logs upon ticket creation," Hazelman said.
Seek expert advice
An emergency situation -- which a complete data backup failure most certainly is -- is best handled by someone with deep experience resolving similar crises.
"Call an expert who can help you weigh any remaining options for recovery," said Ryan Weeks, chief information security officer at backup and recovery vendor Datto, based in Norwalk, Conn. "Most SMBs have an IT service provider or managed service provider that can assist them and, ideally, provide services that avoid such outcomes from happening in the first place."
Even with outside help on board, it may be extremely difficult, perhaps even impossible, to recover lost data.
"Sometimes, some data can be forensically recovered if the backup is partially intact," Weeks said.
A sound business continuity plan assumes all manner of failures and seeks to account for them.
"This is why a robust backup and disaster recovery solution that's resilient to many types of common failures is imperative," Weeks said.
Start testing
It's important to periodically test a backup technology's ability to successfully recover data.
"A backup job may run successfully, but it may not recover successfully. You need to test recoveries," Hazelman stressed.
Remember, too, that something as seemingly innocuous as a security patch, a simple application update or other minor change could inadvertently spark a backup failure, Hazelman said.
"You also need to test any changes you make to all your software, but especially backup software," he added.
In accordance with best practices, always test changes in a sandbox environment before rolling them out to production.
Make sure it doesn't happen again
Just a single experience with a backup failure is generally enough to convince most IT and storage managers to take steps to avoid a repeat performance.
"You should always have a recovery plan in place and test it at least quarterly," Hazelman said.
