Seven ways to keep your cloud storage costs under budget
Cloud storage can be an effective place to house your data, but keeping it cost-efficient means putting in some work up front.
Data storage is a popular use case for cloud computing services, with the rise of "big data" and its value for business intelligence. But without proper management, cloud storage costs can grow faster than your allotted budget, making the business case for storage impractical. There are seven key points to cover in your cloud storage policy to ensure you meet your requirements without over-spending:
1. Categorize your data
Businesses have a wide array of data types that vary in business value. Create a broad set of categories describing different types of data, either by importance -- for example, business crucial, important, optional -- or by functional type -- human resources, financial, research and development. The purpose is to craft policies that meet the needs of each type of data without including unnecessary and costly storage features.
2. Classify storage characteristics
Cheaper storage is not always better, but your policies should help developers and application managers find the most cost-efficient storage that meets their requirements.
There are many ways to store data in the cloud: high or low levels of redundancy, low latency or delayed retrieval times. In addition, some data may be stored for decades and others for just a few weeks. Consider redundancy, retrieval times, duration and availability requirements for each type of data.
3. Define access control policies
One of the most challenging aspects of managing data is keeping it secure long-term. Data stored in the cloud should be organized in ways that allow for streamlined access controls.
Typically, users are organized into groups where each member has similar access requirements. For example, employees in the finance department need read access to historical financial data, but only a few should have write access to the same data. In this case, all members of the finance department may be assigned to a finance read group while the others that need write permissions are also assigned to a finance write group.
Assigning users to groups allows administrators to apply fine-grained access controls so individuals receive only the permissions they need. Groups also facilitate updating permissions. If a new data set is provided to the human resources department, all members of the HR group can be granted access to the data by updating a single group's permission.
4. Consider costs of various storage options
In cloud storage systems, features come with costs. Is high redundancy important? Do you need fast retrieval times? Do your storage requirements entail a minimal number of I/O operations per second (IOPS)? If so, it will probably be more expensive to have more of whatever feature you need. However, you may be able to reduce your cost if you use less feature-rich storage services.
5. Decide which features are best for you
In your policies and procedures, provide guidance on when to use or require particular features. Important documents and data sets should be under version control or review, in order to keep your data organized. Version control maintains a history of changes over the life of the documents or data sets.
6. Consider when to use different types of data stores
You could store data using key-value-like structures using Amazon Simple Storage Service (S3) or in AWS DynamoDB. AWS S3 has a simple pricing scheme based on the amount of data stored and transferred out of the cloud. DynamoDB offers higher performance at a greater cost. These cloud storage costs are based on data size, the number of items read per second, level of consistency (strict consistent is twice as expensive as eventually consistent) and data transferred out.
Cheaper storage is not always better, but your policies should help developers and application managers find the most cost-efficient storage that meets their requirements.
7. Define a data retention policy
As data volumes grow, the question becomes, "How long data should be kept?" Some data might need to be kept in perpetuity for compliance reasons, while other data may have a limited useful lifetime. For example, performance data from a manufacturing device that is no longer in use might have some historical value for comparing to new devices. In this case, it may be more efficient to keep summarized versions of the original data and delete the larger-volume raw data.
About the author:
Dan Sullivan holds a Master of Science degree and is an author, systems architect and consultant with more than 20 years of IT experience. He has had engagements in advanced analytics, systems architecture, database design, enterprise security and business intelligence. He has worked in a broad range of industries, including financial services, manufacturing, pharmaceuticals, software development, government, retail and education. Dan has written extensively about topics that range from data warehousing, cloud computing and advanced analytics to security management, collaboration and text mining.
