2019 IT focus: Storage architecture for big data analytics

Data lake becomes 'swamp'

Chadd Kenney

Chadd Kenney, CTO and VP of product and solutions, Pure Storage: The data lake quickly turned into a data swamp. It ended up becoming a landing zone for a lot of data, and there really weren't a lot of people utilizing the data sets inside of that data lake. I foresee a focus on deploying a data hub that consolidates multiple silos into one common flash-based platform that has high concurrency, high parallelism, and performance that is linear to the scale-out capacity. Big data will be revitalized by the new large-scale data-centric architectures.

Hu Yoshida

Hu Yoshida, CTO, Hitachi Vantara: There's a lot of data that has value that's not even being looked at or processed. Up to now, people have been ingesting it into databases or dumping it into data lakes. But they're coming to the realization that they have to integrate, cleanse, enrich and engineer the data for it to become really useful. This year, the time is right. More tools are becoming available in the cloud and the private sector, and I think people will start to use them. Object stores that have the ability to customize rich metadata are going to be important to intelligently integrate data. The processes are there. The open APIs are there. All those things are working together to enable users to be more intelligent about their storage and their data. It's not just about protecting, replicating, securing and archiving data. It will be more about what the data means to the business.

Scott Sinclair

Scott Sinclair, senior analyst, Enterprise Strategy Group: People have a ton of file data. It's in disaggregated infrastructure. Finding all that data is becoming a real problem. People think of metadata as a way to solve the issue, but the ideal metadata solution has not surfaced yet.

Steven Hill, senior analyst, 451 Research: Companies in the U.S. are only starting to feel the impact of privacy initiatives like [the EU's General Data Privacy Regulation] GDPR. And, with legislation like the California Consumer Privacy Act scheduled to take effect in 2020, businesses will need to extend visibility and governance to both their unstructured data, as well as their traditional database information. Data is now both an asset and a liability, and there is a real need for businesses to better identify and automate the management of unstructured data to remain industry-compliant while continuing to gain ongoing value from the information it holds.

Tuning storage architecture for big data

Storage and memory are merging; compute and memory are merging; compute and storage are merging. At the same time, the networks and interconnects are already at speeds that would be considered pure science fiction less than five years ago.

Steve McDowell, senior analyst, Moor Insights & Strategy: The focus on edge computing, which is one of the hottest areas of IT expansion, has largely been on processing and networking. This changes in 2019, as those deploying these edge solutions start to understand the need for managed storage. The edge, after all, is just a micro data center. This accelerates as data-hungry AI applications are deployed into edge architectures, and as 5G network begins to be seriously deployed in 2019. The serious players in this space must have a storage solution in 2019.

Greg Schulz, senior advisory analyst, StorageIO: For 2019, enterprise data storage focus expands from core cloud and on prem to the edge to support traditional, as well as emerging workloads such as IoT, artificial intelligence, deep learning, machine learning and other analytics. Storage at the edge also means more non-volatile memory (NVM) SSD-based technology, including quad-layer NAND flash, as well as more storage class memory (SCM), such as 3D XPoint. Potential adoption will vary by different IT organizations. Some will expand on what they have done. Others will start looking or do proof-of-concept testing. Also, watch for AWS, Microsoft, Red Hat and VMware, among others, to expand their edge storage solutions and systems.

Phil Bullinger

Phil Bullinger, senior VP and GM, Data Center Systems, Western Digital: The intersection of wireless, networking, storage and compute systems at the edge will accelerate innovation and deployment of new data center architectures. New technologies, including NVMe, composable infrastructures and fabric-attached storage on open standards, and compute architectures tailored for AI and machine learning workloads will displace general-purpose systems as more powerful endpoint devices drive new demands for distributed data infrastructure.

J Michel Metz

J Michel Metz, R&D engineer, Office of the CTO, Cisco Systems: We are in the midst of a paradigm shift in how architectural components relate to one another in the data center. Storage and memory are merging; compute and memory are merging; compute and storage are merging. At the same time, the networks and interconnects are already at speeds that would be considered pure science fiction less than five years ago. In 2019, I expect to see some pretty interesting science experiments from startups that are looking to explode the schematics and reform them into interesting new packaging. For instance, we're already starting to see the combination of storage and compute, called computational storage, storage and networking, and "composable" architectures becoming more of a reality. Once you think out of the box, there will be new and interesting ways to shift the storage functionality around the data center. While there won't be ready-to-play solutions in 2019, you can expect a lot of breakthroughs to be on the market in 2020 or 2021.

Storage management trends

Andy Walls

Andy Walls, IBM fellow and CTO Flash Storage: The use of AI in storage systems will come to fruition in 2019. This is taking advantage of all the data that IBM and Dell and others collect about their storage systems and honing the analytics to be able to find problems before they affect clients and give that information to them in a very consumable way. In 2017 and 2018, for all the suppliers, it was early -- looking at more simplistic things like the configuration isn't quite right or this port isn't connected correctly. Now, as we continue, you're going to start to see more helpful information about some of the trickier problems that occur occasionally, like performance and congestion.

Marc Staimer

Marc Staimer, president, Dragon Slayer Consulting: The ability to move cool and cold data seamlessly from high-performance storage to cloud storage to object storage to file storage -- and not have to rehydrate it to access it -- will become table stakes this year for a lot of storage vendors. In other words, it won't be hierarchical storage management (HSM) type of tiering in the future. You won't have to bring the data back to read it or change it.

Ranga Rangachari

Ranga Rangachari, vice president and general manager, Red Hat Storage: The line between storage and hyper-convergence is getting more and more blurred. What used to be conversations with customers around storage today end up being conversations about hyper-convergence. It's not that they're starting to deploy it today, but as they look at what they need to do over the next 12, 18, 24 months, the need and the desire to build out hyper-convergence environments on prem are becoming more pronounced. People are starting to see the economic benefits of not having to manage the compute and the storage independently and having a hyper-converged way to manage compute, storage and network as one single entity on prem.

Metz, Cisco Systems: Storage management is heading to a crossroads, with different management styles beginning to intersect, including the Storage Management Initiative Specification (SMI-S), NVM Express Management Initiative (NVMe-MI), Redfish, Swordfish and some vendor-proprietary ones as well. 2019 will be a year where these different formats either work well together or collide. Expect to hear about management wars by the end of 2019, and the fight could get nasty.

2019 focus on storage security

Metz, Cisco Systems: Many of the data compliance regulations that emerge out of Europe, Asia, and North America will wind up getting implemented in ways that open up and expose end users far more than they protect them. Storage companies could be left holding the bag, as IoT privacy becomes the siren call for more regulation and more "security." IoT is a data security and privacy nightmare, and recent EU GDPR regulations will have significant unintended consequences as vendors try to play whack-a-mole with all the new threats in 2019.