Storage for edge computing is the next frontier for IoT
IoT edge devices are getting smarter and will need local storage for machine learning and other AI operations. Is the industry ready for the challenge of storage for edge computing?
The history of network storage plays like the bellows of an accordion -- a lot of expansion followed by a lot of contraction.
Networked storage's first "aha!" moments came when system administrators realized they could rope in all that server storage running rampant through the data center doing who knows what. For the first time, they could have some semblance of control -- and security -- over the storage environment.
But it didn't take long for "aha!" to turn into "uh oh" as centralized storage resources grew to overwhelming dimensions. That growth caused management methodologies to meander, backup operations to break and early retirement plans to move forward. The answer, of course, was to decentralize all that storage capacity into little chunks that would be easier to handle.
Too many chunks? Consolidate and crank up that accordion again. That formula seems to have worked for more than a few decades, but 21st century computing is a lot harder to tame. The big game changer is IoT, which adds billions of "things" every year and has become an integral element of digitization for companies across the industrial spectrum.
IoT reshapes traditional IT landscapes
IoT's booming growth is forcing a lot of organizations to rethink traditional IT concepts. As gazillions of devices are added to IoT networks, it's quickly become obvious that more processing needs to take place at the network's extremities. Edge computing eases the burden on more centralized compute resources but, more importantly, reduces the latency induced by moving data back and forth.
If you think this is a niche issue that only huge companies with huge IT budgets are dealing with, think again. "Among the global set of IoT decision-makers we spoke to, 91% have adopted IoT in 2020 (up from 85%), with over eight in 10 having at least one project in the use phase," noted Microsoft's October 2020 IoT Signals report.
That puts a lot of pressure on the edge, but there's little relief in sight as expectations of what can be accomplished there are seemingly boundless. As the Microsoft report states, "AI is the most widely adopted emerging technology -- 79% of organizations adopt AI as part of their IoT solution."
Storage for edge computing
IoT's extremely distributed environments are shaping up to be a nightmare for IT. All that edge computing will need data that needs to be stored somewhere very close to those Tiny Machine Learning (TinyML) chips that add AI to the zillions of sensors, actuators and other devices hanging off the edge. In a 2021 white paper, "TinyML: The Next Big Opportunity in Tech," ABI Research predicted "the TinyML market will grow from 15.2 million shipments in 2020 to 2.5 billion in 2030." That's a lot of AI processing.
Instead of having to deal with hundreds or even thousands of VMs and the storage they need, IoT edge could mean tens or hundreds of thousands of devices that need local storage -- it's like decentralization on steroids.
Storage vendors -- it's time for you to step up and to apply all that networked storage expertise to storage for edge computing: "To boldly go where no storage vendor has gone before." (OK, so they don't say storage vendor on Star Trek, but you get the idea.)
Edge storage might not be so easy
Putting high-performance storage in thousands -- or millions -- of places and then managing the whole deal is something of a tall order.
First, any single IoT may have dozens or even hundreds of different types of devices adorning its edges. Each device may interact differently. Communications are likely to use protocols unfamiliar to the storage world, such as MQ Telemetry Transport, Advanced Message Queuing Protocol, 4G and 5G LTE, and a variety of short-range wireless protocols.
The hardware part of the edge storage issue appears to be under control, but the firmware and software that will be needed to keep up with processing and storage speeds will require more development.
The processors embedded in the edge devices will also vary. This means the way a Raspberry Pi accesses data from storage might be different from the way a Banana Pi or an Onion Omega2 does it.
Power may be an issue. We're not going to spin up disks at the edge, but even solid-state's modest power appetite might seem gluttonous when multiplied by thousands of edge instances.
Today, most storage for edge computing comes in the SD and microSD formats, which provide ample capacity and currently seem to be quick enough to handle AI chores. New forms of solid-state storage are sure to emerge that are cheaper, faster and consume less power.
The hardware part of the edge storage issue appears to be under control, but the firmware and software that will be needed to keep up with processing and storage speeds will require more development. The biggest challenge might be managing all that storage. There won't be huge capacities to worry about, but with so many individual instances organizations must configure, protect and back up, managing an IoT edge storage environment won't be easy. And moving data from edge devices to cloud services and data centers could devolve into a mind-bending data traffic jam.
