Part of:Track outside-the-box storage technologies
Holographic data storage: Next big thing or bust?
The latest work in holographic storage development approaches its uses in a different way. Take a deep dive into the possibilities and potential of this 3D storage.
Holographic data storage has been discussed for decades. Once slated as the next generation of optical storage, it promised greater densities and access speeds than today's Blu-ray discs.
Over the years, numerous research teams tried to build holographic systems that could meet the growing demands of data storage. However, these teams achieved few concrete results beyond the occasional prototype. Their efforts were not in vain, however. Microsoft breathed new life into holographic storage with Project HSD, a collaboration between Microsoft Research Cambridge and Microsoft Azure, whose goal is to adapt holographic technology to cloud-scale storage.
Why holograms for storage?
Holographic storage -- sometimes referred to as 3D storage -- is a volumetric storage system that uses lasers to read and write data, similar to other optical storage. However, media such as CDs, DVDs and hard disks can store data only on the medium's surface, limiting its capacity to two-dimensional storage. Holographic storage uses the entire volume, which makes it possible to store more data in a smaller space and increase data write and read speeds.
Polaroid researcher Pieter J. van Heerden first proposed holographic data storage in the early 1960s, not long after the invention of the laser. By the early 2000s, research teams in both industry and academia had made significant progress in demonstrating the technology's potential. Two prominent efforts came from Polaroid spinoff Aprilis and Bell Labs spinoff InPhase Technologies. Both companies tried to bring holographic storage to the market. In the end, however, neither one achieved commercial success. Dow Corning acquired Aprilis and InPhase eventually filed for bankruptcy.
There were many other efforts as well, but none could turn the tide on holographic data storage. Most of these attempts focused on the use of circular media similar to CDs or DVDs to support write-once, read-many (WORM) operations, but holographic storage competed against more established technologies, which themselves had also advanced.
HDDs, for example, became faster and denser, and SSDs got cheaper and more durable. At the same time, there was increased reliance on cloud computing, which brought scalable storage and extensive streaming capabilities.
Despite these trends, the need for innovative storage platforms continues to grow. According to Microsoft, the world will generate 125 zettabytes of data annually by 2024. Enterprises and cloud service providers must devise economical ways to store this data and meet its necessary performance, availability and durability requirements.
Current storage technologies are insufficient to maintain this volume of data. HDDs, for example, are limited by their mechanical nature. SSDs are still relatively expensive to implement at scale and don't always deliver the necessary endurance.
Project HSD takes on holographic storage
To help address future storage needs, Microsoft launched Project Holographic Storage Device (HSD), a collaborative research effort that revisits the holographic technology, but this time with the idea of providing cloud-scale storage to support warm data.
Project HSD is part of Microsoft's Optics for the Cloud group at Microsoft Research Lab in Cambridge, England. Another one of the group's efforts is Project Silica, which experiments with the use of crystals to deliver long-term archival storage. However, Project Silica focuses on WORM operations only, like traditional approaches to holographic data storage. Project HSD makes it possible to erase and rewrite data and will offer faster read and write throughputs.
According to Microsoft, the project's mission is to "design mechanical-movement free, high-endurance cloud storage that is both performant and cost-effective." Microsoft also states that the project has already achieved density 1.8 times higher than earlier volumetric holographic data storage. The team works to increase densities even further and achieve faster access rates.
To help deliver on these promises, Project HSD uses commodity components such as the high-resolution cameras and display screens in today's smartphones. The project also uses machine learning and deep learning to further improve precision and performance. As a result, the team has reduced optical distortions and manufacturing tolerance requirements. It uses software to compensate and calibrate the system at runtime.
The material used for the storage medium also sets Project HSD apart from other attempts at holographic data storage. Many of the other projects used polymers to store permanent changes in the material, which is why they were limited to WORM operations.
In contrast, Project HSD stores the holograms in electro-optic crystalline materials. The project stores each hologram as a spatial variation in the distribution of the electron density, which it can change by exposing the medium to light of a specific wavelength. The holograms can also be erased by exposing the crystalline material to ultraviolet light.
Despite Microsoft's departure from traditional holographic storage, the basic approach to reading and writing data is much the same. The storage process begins by splitting a laser beam into two signals. One of the beams carries the data to the storage medium. The data-carrying beam -- also referred to as the data, object or signal beam -- passes through a device called a special light modulator, which then passes or blocks light at points corresponding to binary 1s and 0s. The modulated data beam then continues to the crystalline material.
The second light beam is referred to as the reference beam. The beam does not pass through a light modulator but bounces off a mirror and is redirected toward the storage medium, where it intersects with the data beam to create a 3D interference pattern in the optical material.
The pattern forms a tiny hologram that represents a single data page, which can hold hundreds of kilobytes of data. A data page occupies a small volume, or zone, within the optical material. A zone can contain multiple pages, and the storage medium can contain multiple zones.
A holographic storage device reads data by diffracting the reference beam off the hologram in the storage medium. The data beam is not required for this operation. A camera captures the diffracted image, which makes it possible to reconstruct the original data page. A holographic storage system can read different holograms by changing the angle of the reference beam, or it can erase the holograms with UV light, making it possible to rewrite data.
Holographic data storage promises a cost-effective way to answer specific business questions in an efficient and timely manner.
Uses for holographic data storage
In its use of crystalline materials, Project HSD takes advantage of the inherent parallelism in optics. It enables data to write to and read from the storage medium in parallel, which results in higher overall throughputs. The project's approach also requires fewer mechanical parts, such as those found in HDDs. Instead, it limits movement to readjusting the angle of the laser beam; all other components remain fixed. In addition, holographic storage can use the medium's entire volume, rather than just its surface, which offers greater densities than current types of optical storage.
Traditional approaches to holographic storage focused on data archiving and support of WORM operations. Project HSD targets warm data that supports both read and write operations, which could benefit cloud providers and enterprise data centers. Warm data is typically accessed and updated less frequently than data that supports important business applications and is seldom maintained in real time, although it usually requires greater scalability. Performance requirements can vary, depending on the supported workloads.
Holographic data storage promises a cost-effective way to answer specific business questions in an efficient and timely manner. Its fast read performance and ability to update data make it well suited to data warehousing, big data analytics and applications that incorporate advanced technologies such as predictive analytics or artificial intelligence.
Organizations that need to generate regular reports -- such as weekly call-center statistics or monthly sales figures -- could benefit from holographic data storage. Holographic storage could also support less critical operations, such as providing support personnel with the background information they need to help their customers.
How close are we to holographic storage?
Despite its promise, holographic storage has a long way to go to get from the research phase to the point where enterprises can purchase commercial products. Manufacturers would have to set up entirely new environments for building the storage devices, which will require a high degree of precision to ensure proper alignment between components. In addition, most of the research on Project HSD has focused on writing to and reading from a single zone. The team still faces the challenge of delivering the same level of performance across multiple zones. Another concern is to ensure that inadvertent exposure to UV light does not erase the data.
Because holographic data storage is a technology with so many false starts, it's no surprise that Microsoft has avoided making predictions about the technology's commercial application. In the meantime, there are plenty of other efforts to keep the industry entertained, ranging from storage class memory to DNA storage.
