Economic impact of large capacity archival flash
Quad level cell flash technology is leading to high capacity flash devices that have applicability in archival storage or content repositories that are primarily read access. Currently at 96 layers, QLC devices are different than the flash devices used in primary storage systems. The opportunity for this technology in archival storage and content repositories has led to the term “archival flash.”
Toshiba, Samsung, and Intel will most likely be the first vendors delivering the archival flash devices, and other flash device vendors may follow with their own devices. Archival flash devices are projected to have capacities in the 100 TB range – far higher than almost any SSDs on the market now. While Nimbus Data already ships 100 TB SSDs, it is not using QLC and may not have the much lower cost of the newer technology.
The large capacity is warranted given the primary use for storing archival data. Some in the industry remember the issues in primary storage when disks increased in capacity, raising concerns about rebuild times. Those familiar with the protection from device failures used in object storage systems — the most likely systems to use archival flash — will understand how the circumstances are different. Object storage systems protect device failures through information dispersal algorithms and erasure codes within a node. Node failures are protected with N+1 node protection using data distributed across those nodes. A site is protected with either replication or geographic dispersion, adding another level of protection to the immutable data stored in object storage systems.
Archival flash economics
Unfortunately, some still look at the economics of storing data with a one-dimensional view of data-at-rest economics with the focus on acquisition costs. Using data-at-rest economics effectively says that all data is equal and low cost is the only value a user would achieve from acquiring a system. It ignores different types of devices that have many different attributes. With achieving value from performance that has been demonstrated with current SSDs (not using QLC technology), data-at-rest economics should have been discounted. For archival flash devices, the main value for customers will be in the longevity of the QLC devices. Having 12 to 15 years of expected lifespan, the longevity changes the value for customers.
Data has gravity, meaning that data stored for a period of time tends to persist and incurs costs to move to new systems. Archival flash longevity will invalidate the one-dimensional acquisition cost economics and require evaluation of TCO that include technology lifespan.
Longevity requires the storage system to have the capability to disaggregate the storage devices in an enclosure independent of the controller function. Disaggregation allows the data to remain in place on the longer lifespan devices while the controller is updated based on the technology change rate for processor, adapters, etc. in the controller. Many vendors have already accomplished this and feature the capability with “Evergreen” programs. This allows the economics of the different technology change rates to be optimized.
Economic modeling
To show the effect of the longevity of archival flash used in an archival storage system, let’s look at an economic model to show the total cost of ownership over a 15-year time span. The model includes a wide variety of parameters and what-if scenarios:
- Initial capacity
- Annual capacity growth rate
- Lifespan of archival flash device and lifespan of standard storage device
- Capacity of archival flash device and capacity of standard storage device
- Costs and requirements used in TCO calculations for administration, deployment, space, power, rack, and cables.
- Average yearly price declines for archival flash device and standard storage device
- Average yearly price decline for controller/node
- Number of devices attached to controller/node
- Hardware and software average discounts
- Controller/node costs
- Software costs – capex and software capacity-based license charges
- Initial per GB price of archival flash and standard storage devices
- Device level – protection – number of devices data is distributed across and number of segments of data protection (for example – Archival flash default of 12 devices with toleration of three devices failures in addition to protection of data distributed across N nodes). For standard devices the example would be 8 devices with two used protection.
The first diagram shows the comparison with capacity and longevity values, which are shown on the right side. This diagram has the acquisition costs of the archival flash to be 50% higher than that of the standard devices, which in this case are large capacity disks. For simplicity, price declines are set to zero for this example. Object storage systems capable of retiring a node and automatically redistributing data is assumed, which will avoid migration costs that would be evident in other storage systems. In this model diagram, data reduction was enabled for both types of devices with equivalent effectivity. Discounts were at 30% of list price. Premium support services were not added to the costs. Technology transitions occur in the final year of the lifespan, accounting for overlap in the technologies.
It should be clear that the longevity of archival flash has a dramatic effect compared to the shorter lifespan of standard devices. This also illustrates the inadequacy of using a one-dimensional, data-at-rest cost of storage measure. Savings come from having to purchase fewer replacement devices with archival flash over the 15-year span because of the longer lifespan and from the higher capacity with archival flash requiring fewer nodes as the system scales. Increasing capacity of devices over time was not included in the projection because of the limited history with archival flash devices.
The following diagram shows where the economic savings are realized:
The chart shows significant cost differences, with the largest contribution coming from the avoidance of acquiring replacement devices.
Looking at the “what-ifs,” the following chart includes changes in some parameters. Specifically, the device prices were set to be equal and the lifespan of archival flash was set to 15 years.
Another “what-if” adds the price decline experienced over the last 5 years. Those results show another 8% improvement in costs for archival flash over standard devices.
Given that a large amount of data is retained “forever” in archive and content repository systems, the economics of the large capacity requirements will impact archival flash.
It will be difficult to overcome initial thoughts such as the one-dimensional data-at-rest economics and limited understanding of data protection for large capacity devices used in object storage systems. But that will only delay the rise of archival flash for storing and managing data.