Data growth spawns enterprise data management system challenges
Enterprises are creating and using more data and storing more of it in the cloud. If they use that data strategically, they can improve business outcomes.
Organizations are creating and consuming more data than ever before, spawning enterprise data management system challenges and opportunities.
A key challenge is volume. With enterprises creating more data, they need to manage and store more data. Organizations are now also increasingly relying on the cloud for enterprise data management system storage needs because of the cloud's scalability and low cost.
IDC's Global DataSphere Forecast currently estimates that in 2020, enterprises will create and capture 6.4 zettabytes of new data. In terms of what types of new data is being created, productivity data -- or operational, customer and sales data and embedded data -- is the fastest-growing category, according to IDC.
"Productivity data encompasses most of the data we create on our PCs, in enterprise servers or on scientific computers," said John Rydning, research vice president for IDC's Global DataSphere.
Productivity data also includes data captured by sensors embedded in industrial devices and endpoints, which can be leveraged by an organization to reduce costs or increase revenue.
Rydning also noted that IDC is seeing growth in productivity-related metadata, which provides additional data about the captured or created data that can be used to enable deeper analysis.
Ranking organizations by data maturity, an Enterprise Strategy Group survey sponsored by Splunk found that few organizations are data innovators.
Enterprise data management system challenges in a world of data growth
Looking ahead, Rydning sees challenges for enterprise data management.
Perhaps the biggest is dealing with the growing volume of archived data. With archival data, organizations will need to decide whether that data is best kept on relatively accessible storage systems for artificial intelligence analysis, or if it is more economical to move the data to lower-cost media such as tape, which is less readily available for analysis.
Another challenge is handling data from the edge of the network, which is expected to grow in the coming years. There too the question will be where organizations should store reference data for rapid analysis.
"Organizations will increasingly need to be prepared to keep up with the growth of data being generated across a wider variety of endpoint devices feeding workflows and business processes," Rydning said.
The data management challenge in the cloud
In 2019, 34% of enterprise data was stored in the cloud. By 2024, IDC expects that 51% of enterprise data will be stored in the cloud.
While the cloud offers organizations a more scalable and often easier way to store data than on-premises approaches, not all that data has the same value.
Companies are continuing to dump data into storage without thinking about the applications that need to consume it.
Monte ZwebenCo-founder and CEO, Splice Machine
"Companies are continuing to dump data into storage without thinking about the applications that need to consume it," said Monte Zweben, co-founder and CEO of Splice Machine. "They just substituted cheap cloud storage, and they continue to not curate it or transform it to be useful. It is now a cloud data swamp."
The San Francisco-based vendor develops a distributed SQL relational database management system with integrated machine learning capabilities. While simply dumping data into the cloud isn't a good idea, that doesn't mean Zweben is opposed to the idea of cloud storage.
Indeed, Zweben suggested that organizations use the cloud, since cloud storage is relatively cheap. The key is to make sure that instead of just dumping data, enterprises find way to use that data effectively.
"You may later realize you need to train ML [machine learning] models on data that you previously did not think was useful," Zweben said.
Enterprise data management system lessons from data innovators
"Without a doubt, some companies are storing a lot of low-value data in the cloud," said Andi Mann, chief technology advocate at Splunk, an information security and event management vendor. "But it is tough to say any specific dataset is unnecessary for any given business."
In his view, the problem isn't necessarily storing data that isn't needed, but rather storing data that isn't being used effectively.
Splunk sponsored a March 2019 study conducted by Enterprise Strategy Group (ESG) about the value of data. The report, based on responses from 1,350 business and IT decision-makers, segments users by data maturity levels, with "data innovators" being the top category.
"While many organizations do have vast amounts of data -- and that might put them in the data innovator category -- the real difference between data innovators and the rest is not how much data they have, but how well they enable their business to access and use it," Mann said.
Among the findings in the report is that 88% of data innovators employ highly skilled data investigators. However, even skilled people are not enough, so 85% of these innovative enterprises use best-of-breed analytics tools, and make sure to provide easy access to them.
"Instead of considering any data unnecessary, look at how to store even low-value data in a way that is both cost-effective, while allowing you to surface important insights if or when you need to," Mann suggested. "The key is to treat data according to its potential value, while always being ready to reevaluate that value."
_mobile.jpg)
