Government heads to cloud for excess archival storage needs
Boone County in Indiana found space and savings in public cloud, but experts say IT pros should follow smart policies and government regulations when managing vital data.
Government organizations are finding their storage needs increasing, exacerbated by employees working from home due to COVID-19 lockdowns. They're also seeing an exponential rise in unstructured data from state surveillance devices such as police body cameras.
Some municipal organizations are heading to the cloud for their archives, particularly for storing surveillance footage, while others have doubled down on fast on-premises systems. Uses and needs can vary due to laws and regulations for public over private data, but protecting and storing data remains mission-critical to local governments.
Failure to do so can cause innumerable issues. Earlier this month, the Dallas Police Department made headlines when it acknowledged the department permanently lost 8 TB of data earlier this year, after a failed migration from the cloud to an on-premises server.
The loss, originally tallied at 22 GB deleted over several days, included images, documents, videos and other data collected by the department. Some of the lost data included evidence for active criminal cases, resulting in trial delays and released suspects. Department officials have since placed the blame on human error.
Proper policies and safeguards can reduce issues that arise, but government IT professionals are among the vanguard of exponential data growth and management, according to experts.
"Everybody who has a lot of data to manage is overwhelmed," said Marc Staimer, president of Dragon Slayer Consulting. "We're at the beginning of a tsunami of data. We're talking about a once-in-a-million-years tsunami."
Making investments
Contracted IT services for Boone County, Ind., decided to invest in archival cloud storage on Azure in 2018 to house a massive amount of data generated by the sheriff's department.
The department covers five towns, numerous unincorporated areas and one populous city and had begun issuing body cameras for officers. Additionally, the county's court system required storage of files relating to ongoing and closed cases and had to factor in other police departments' body camera files.
Boone County storage needs on Azure have increased from 25 TB to 35 TB in the past three years, according to the county's IT contractor, Government Utilities Technology Service (GUTS). As more departments across the county begin utilizing cameras, GUTS contractors said they expect those needs to keep increasing.
"Every type of case has a different retention schedule," said Sean Horan, a manager of IT services for Boone County at GUTS. "There are some which only have a four- or five-year retention cycle. There are others that are forever. The problem for us as an IT group is we cannot sit down and say we can delete X or Y, because we don't know."
When you put that into perspective for what that would cost with SANs, it's a no-brainer.
Sean HoranManager of IT services for Boone County, Ind., Government Utilities Technology Service
Data management software by Komprise helps sort what data heads to archive storage or what remains on premises, Horan said. He estimated that annual bills with Azure total $25,000 to $30,000 but come out significantly cheaper compared with the on-prem-exclusive mentality of prior decades.
"When you put that into perspective for what that would cost with SANs, it's a no-brainer," Horan said.
Data stored on Azure by the county is also backed up on tape.
Previously, Horan and his team had considered making their on-premises storage entirely flash storage but settled on a hybrid flash and hard drive array totaling 55 TB from Dell EMC. The all-flash configurations he investigated totaled almost $1 million, Horan said, while the hybrid approach rang up to a more reasonable $125,000.
Still, constant expansion into the cloud isn't the all-encompassing storage product for Boone County, Horan added. Instead, smart use of devices can help cut down on the need for redundant recordings and data generations.
"It just snowballs," he said. "Bodycams are a good thing, don't get me wrong, but it's not just the storage. We always try to look at ways to improve processes."
Government organizations have always seen the value in storing data with the cloud, but regulations typically limit who, where and what sort of storage business they can work with, said Dave Raffo, an analyst at Evaluator Group.
"The government was an early proponent of the cloud," Raffo said. "They're following the same thing we're seeing in the enterprise. The government has a lot of extremes, whether it's performance or capacity."
Those regulations mean companies whitelisted through contracts and bids typically have a very lucrative market open to them, Raffo said.
VAST Data recently announced that it received $10 million in orders from the U.S. Department of Defense (DoD) through its government tech subsidiary, VAST Federal, for its on-premises storage and private cloud offering. Meanwhile, cloud hyperscalers including AWS, Microsoft, Oracle and Google became ensnared in legal battles over the former $10 billion Joint Enterprise Defense Infrastructure contract also put forward by the DoD.
"There are a lot of vendors targeting that market," Raffo said.
