Block storage comes with challenges, including high data egress costs, confused billing, inconsistent performance and availability issues. Find out how to cope with the top four.
A recent Taneja Group survey found that two-thirds of the respondents use cloud-based block storage services, up significantly from just a few years earlier. At the same time, respondents also reported several challenges with these services.
The top four challenges are high data egress costs, confused billing, inconsistent performance and downtime and availability issues. These challenges might not be enough to dissuade organizations from using cloud block storage services, but it's important to be aware of them and prepare accordingly before committing to a block storage solution.
1. High data egress costs
Providers offering cloud block storage services generally don't charge data ingress fees, that is, they don't charge for moving data onto their platforms. However, most do charge for data egress, essentially penalizing customers for copying their data to other cloud services or back to their own data centers. Worse still, customers are often unprepared for the enormity of these charges, failing to fully understand the cloud storage costs before signing on to a service.
Consider the following example. You're launching a big data analytics project on a platform other than the one you're using for block storage. The project requires an initial 25 TB of data to be copied from the block storage platform to the project's platform. This step alone could run thousands of dollars beyond the basic storage costs. And it doesn't stop there. After the initial data load, the project will require weekly data transfers, with additional fees for each one.
Organizations that plan to implement cloud-based block storage should factor in these egress charges when selecting a platform and planning application workflows. For example, it might make sense to optimize applications to reduce data transfers, keeping more active data on premises or limiting the use of cloud storage to certain applications where the data is likely to stay on that platform.
2. Confused billing
Cloud service providers are notorious for their confusing and complex billing, and it's no different with block storage. Not only does this apply to unexpected egress charges, but also to other storage-related charges, which can add up fast and often be difficult to decipher and predict, especially as workloads evolve and more teams within an organization make use of the storage services.
Part of the challenge is providers routinely change their subscription fees and pricing structures, making it difficult to keep up with current charges and nearly impossible to predict long-range costs. And these issues become more complex as you add other cloud services to the mix. Organizations working with multiple providers have an even harder time as they navigate the different ways that providers name and structure their services.
Customers must be proactive when it comes to navigating cost complexities. They should dedicate the internal resources necessary to manage and optimize costs, using a platform's cost calculators or third-party tools to make sense of the billing and keep it under control. Customers should also use platform features such as tagging to tie specific storage services to the teams that use them.
3. Inconsistent performance
One of the top challenges of many cloud block storage services is inconsistent performance, which can be particularly problematic for the types of applications that use block storage, such as relational database management systems and virtual desktop infrastructures. These applications require consistent and dependable I/O and throughput and don't handle disruptions that impact business operations well.
These challenges might not be enough to dissuade organizations from using cloud block storage services, but it's important to be aware of them and prepare accordingly before committing to a block storage solution.
One reason for performance issues is that cloud-based block storage is typically implemented as a SAN, which itself comes with limitations. For instance, data traveling across networks can result in higher latency rates compared with DAS. Plus, data traffic might have to contend with other storage or network traffic, which is complicated even more by the cloud's shared architecture. Because block storage is typically virtualized, VM configurations can limit the maximum IOPS or throughput and create performance bottlenecks.
To address performance issues, customers should start by monitoring their storage infrastructures. For instance, Microsoft Azure customers can use Azure Monitor to view disk-level metrics in order to isolate performance issues. They might also consider deploying multiple disks and then striping them to achieve more predictable performance.
VM types and configurations can make a difference. For example, Amazon offers instances optimized for Elastic Block Store (EBS) and that provide dedicated bandwidth between the Elastic Compute Cloud and EBS platforms. The steps you take to optimize storage often will be specific to your data storage platform.
4. Downtime and availability issues
Other challenges with cloud block storage services are unplanned downtime and unpredictable availability, either of which can severely impact critical workloads. A transaction-intensive application might use a block storage service for its data requirements. If the data is unavailable, operations will come to a halt, leaving users without access to their information and rendering the application temporarily useless.
Downtime and cloud availability issues can occur for several reasons. For example, if two AWS instances use the same EBS volume name, the second instance might try to attach to the volume before the first instance releases it. In addition, issues can sometimes arise if a VM tries to mount or unmount a volume, even if there are no naming conflicts. Customers can also run into problems when the provider tries to update the underlying infrastructure or when there's a power failure and backup systems don't kick in properly. In fact, a power failure can potentially lead to permanent data loss.
Addressing such issues can be difficult for customers because many of them originate with the provider's infrastructure and processes, which are out of the customer's control. For this reason, IT should vet each provider and review available service-level agreements to ensure the platform can meet workload requirements. IT should also keep availability in mind when setting up the service. For example, Google's Persistent Disk service lets you use regional persistent disks to ensure higher availability. In addition, IT should implement a backup strategy that fully protects data against provider mishaps.
Beyond the top four cloud block storage challenges
The Taneja survey points to several other challenges with block storage services in addition to the four covered here, including issues with application syncing and replication, slow backups and snapshots, and limited scalability. The survey also indicates that 22% of respondents reported no significant issues, a promising trend.
In addition, the top two issues, data egress costs and confused billing, are business-related concerns, rather than technical ones, suggesting that cloud storage is becoming more mature. With luck, cloud providers will come up with more reasonable pricing approaches, before customers look for other options.
