IoT data management extends best practices to the edge
IoT devices generate and collect data from points all over the network. Organizations must apply general data management best practices to get the most value from IoT data.
To use IoT-generated data and keep it secure, organizations must extend their data management practices to the edge.
IDC predicts the volume of data generated at the edge will experience a 34% compound annual growth rate each year through 2027. That rate is faster than the growth expected for data generated at the core or other endpoints. To protect and use all that data, security and data management practices must reach beyond core enterprise systems and business applications to where the data is being generated.
Strong and secure data management practices ensure that an organization can successfully ingest, store, organize, maintain and protect its data assets. Responsible management leads to more data insights and better decision-making. It can also power AI and advanced analytics systems.
IoT data management challenges
Experts said that data leaders face a host of challenges around IoT data management, including identifying all data sources, integrating data, ensuring the quality of the data and applying the right controls to match the data's security needs.
The IoT environment also presents unique challenges, including the scale and volume of IoT-generated data. The amount of IoT data is enormous, and IoT devices constantly record and add more data points.
"There has been an overwhelming tidal wave of new IoT applications, and so the volume of data, and the level of granularity of the data, is way beyond what any IT person was trained to expect," said Nicholas Napp, a senior member of the IEEE and co-founder of Xmark Labs.
Nicholas Napp
Data and technical leaders struggle to understand that organizations can use each of the varying data points to produce business benefits. They also struggle with creating and implementing the infrastructure needed to store and use all that IoT data.
Another significant challenge with IoT data management stems from IoT's distributed nature, Napp said.
"Roll the clock back 10 or 15 years, when enterprise IT really understood the ins and outs of its network and the systems on it. Now practically anybody can have an IoT device," he said.
Managing IoT devices
Data and IT leaders face more work cataloging all the IoT devices deployed in their environments, understanding what data is produced and devising strategies on how to manage data sets.
The operations teams and non-technical workers who frequently deploy IoT devices are not usually as knowledgeable about security and privacy requirements as the data leaders. They don't incorporate appropriate security and privacy controls from the start, Napp said. He pointed to the ease of implementing smart cameras as an example. Facilities teams commonly implement smart cameras, but don't know how to secure the data coming off the cameras.
The number of [IoT device] protocols is one of the challenges around IoT data that makes IoT data management unique and more challenging.
Tancred TaylorSenior IoT analyst, ABI Research
"Everything has a vulnerability, and if you're unfamiliar with the technology, that is a potential risk from a data management point of view," Napp said. As a result of that unfamiliarity, data teams often have to go in after deployments and retroactively identify and implement required controls.
"The number of protocols is one of the challenges around IoT data that makes IoT data management unique and more challenging," said Tancred Taylor, senior IoT analyst at ABI Research. The varying protocols and the need to ingest data into a single data engine means "you need a lot of drivers and data translation tools so that they all [eventually] speak the same language."
IoT devices generate unstructured data that does not make much sense on its own, Taylor said. For example, there are IoT devices that record and transmit data on the level of vibrations produced by manufacturing equipment at set times.
"That data doesn't make sense on its own, so it has to undergo some deep analysis or be mixed with enriched data so companies know what they should do with that data," he said.
The last challenge experts mentioned is the inventorying of all the data sets generated by IoT devices . Understanding which parts of data are important and which aren't is a big task, said Sarb Sembhi, a member of the Emerging Trends Working Group at ISACA and CTO of Virtually Informed.
AI, machine learning and other cutting-edge technologies can plumb IoT data to expose more of its potential and produce new business value.
Best practices for IoT data management
Experts advise organizations to have a solid data management function for handling IoT challenges:
Secure the data according to organizational and regulatory requirements.
Formulate data quality policies and practices.
Implement policies and practices for integrating, storing and maintaining data.
Engineer systems to deliver the right data to the right users, whether human workers or applications, at the right time.
Once a data management function is in place, data leaders must layer in additional policies and practices to address the specific needs of IoT data.
Establish data security
Understand the security features and limits of the various IoT devices, protocols and connecting networks. Use that knowledge to create additional security policies and practices to ensure the security of data within the IoT environment matches those established for data within the core IT environment.
"Understand all the pieces of your IoT stack -- what's being communicated, how it's being communicated, and the vulnerabilities in each step from the IoT device to the data's final resting place," Napp said. "You need to look at every step and make sure somebody really understands what's going on at each step."
Analyze data
Identify what IoT data is useful for what purposes. Identify what data is enough on its own for those purposes and what needs augmentation with additional data to be useful, Taylor said.
Process data
Determine what data needs edge processing to meet use case requirements -- such as for real-time monitoring and alerts -- and what data centralized systems can process and store, whether in the cloud or in on-premises data centers.
Continually assess data
Revisit and reassess what IoT data can stay at the edge and what to send to a central data system. Advancements bring more compute power to edge devices, making TinyML more mainstream and allowing for more analytics closer to the IoT devices, Taylor said. AI -- particularly generative AI -- helps data leaders understand and use data.
Integrate data
Combine IoT data with other organizational data so all of it is available for use and distribution to the right people at the right time, Taylor said.
Mary K. Pratt is an award-winning freelance journalist with a focus on covering enterprise IT and cybersecurity management.
Nicholas Napp
