Evaluating IoT data storage in the cloud
Organizations increasingly need to store data generated from the Internet of Things, which means deciding between cloud and on-premises IoT data storage is more important than ever.
The term Internet of Things (IoT) describes the various devices and sensors connected to the Internet and their ability to capture and transmit data. Organizations use IoT devices to make better business decisions and improve customer satisfaction. But after this data has been transferred, it needs to be amassed in a storage system -- which is causing companies to rethink their data storage infrastructures.
While the cloud seems to be an obvious choice for IoT data storage, many organizations keep this information on site, believing it is either too sensitive or too expensive to store in the cloud. But before you can discuss the attributes of your data storage environment, you need to pin down where this data will be stored.
The cloud has several advantages over on-premises storage for IoT data:
- There is typically a more direct connection between the device and the public cloud provider. This direct link means data can be stored off-device faster, resulting in less storage on the device and lower per-device cost.
- Storage management is now the cloud provider's problem. The provider's job is to provide a service, and the organization just needs to use that service.
- The cloud makes an ideal storage location if the organization is using cloud compute to process IoT data.
Using cloud storage beyond backup
But there are concerns with using the cloud for IoT data storage. The biggest one is security. In many cases the data being captured is legally sensitive or proprietary to the organization. The chance of that data being accessed inappropriately due to a security breach is a legitimate concern, but the reality is that most cloud providers have excellent security or at least provide the organization with the tools they need to secure their data. In most cases, breaches are caused by human errors, not by a weakness in provider security.
The second is more pressing: the cost to store the IoT data set. While cloud storage costs are impressive when considered on a per-gigabyte (GB), per-month basis, the allure wears off when the math is done to calculate storing petabytes of information for decades. If the decision is made to store data in the cloud, there are several attributes of cloud storage providers to consider beyond the price per GB per month:
- The overall cost, which includes total capacity (factoring growth), multiplied by the number of years to maintain.
- The cost to transfer data in and out of the cloud provider. Most providers do not charge for data as it comes into the cloud, but many charge a bandwidth fee as it leaves. The transfer consideration is especially important if the compute will be done on-premises, which means some transfers will occur no matter what.
On-premises IoT data storage considerations
If IoT data is stored on-site, the next decision to make is whether to use a traditional NAS array or a private cloud architecture. The NAS array has the advantage of familiarity, but often lacks cost-effective scaling and modern protocol support such as objects and Amazon S3. The primary objective of an on-premises store is to ensure data is stored cost-effectively but reliably and with minimal staffing. The on-premises strategy should be less expensive for long-term storage of large data sets, but because it will clearly become an IT function, the organization needs to be prepared to make that commitment.
The sole purpose of most devices that fall under the IoT umbrella is to capture data. That means IoT will affect storage decisions for years to come. If data is stored on-site, you should start building that infrastructure now. IoT data is growing rapidly and the rate of growth, as well as the time that organizations want to keep the data, will only accelerate.
