How to get the full benefits from NVMe flash storage
Consolidate and increase the density of your organization's workloads in a flash storage system to take full advantage of NVMe's parallelism and reduced latency.
Nonvolatile memory express is a new standard designed for flash and memory-based storage. As all-flash arrays start to support this new technology, vendors will swap SAS-based flash drives with NVMe-based drives. However, the SAS-based, all-flash systems in most data centers already provide more performance than required. In order to take full advantage of NVMe flash storage, data centers must rethink the design of those systems and consider implementing new techniques such as quality of service.
The primary impact of NVMe in an all-flash system is not necessarily an increase in raw performance. NVMe is an interface and protocol, while the flash on the drive is still the same as the flash on a SAS-based drive. What NVMe provides is latency reduction. NVMe supports a much higher number of command queues than SCSI and more commands per queue. The net effect is NVMe flash storage can handle more workloads and a greater variety of them at the same time.
Designing for NVMe
Conventional wisdom targets NVMe-based systems at database applications like Oracle and Microsoft SQL, which are perfect candidates because of the hundreds of users reading and writing data at the same time. NVMe lets these applications scale user count and the amount of data queried without having to change the server hardware or the software licensing. Essentially, NVMe-based flash systems allow the database to do more with the same software and hardware. The problem is few organizations have database environments that need to scale to these new levels.
To take full advantage of NVMe flash storage, businesses must look elsewhere and add additional load to the system. Virtual workloads are a good place to start.
Most virtual environments have a relatively low virtual machine to physical server density, although there is plenty of CPU power to support more VMs. In the past, the concern with increasing VM density was creating a bottleneck in the network or on the storage system. With higher speed and better-designed networks, this has become less of an issue. Consequently, with NVMe flash, the all-flash array can respond quickly to the multitude of I/O requests highly dense VM environments create.
If you buy an NVMe system, you should look to consolidate as many workloads as possible on it, including multiple databases and hypervisors and other modern workloads, such as big data analytics and container-based-environments.
Overcoming consolidation risk
With an NVMe-based all-flash array, storage consolidation can scale to a level previously considered impossible. However, as with any consolidation effort, there are risks. For instance, putting a lot of workloads on one system can be issue. Using a high-quality system with appropriate high-availability capabilities can mitigate this risk.
To realize the benefits of NVMe, enterprises must consolidate more workloads on the storage system and increase the density of those workloads.
A more significant concern is how to handle peak loads. Consolidation of the magnitude NVMe provides leads to the possibility for two or more applications to spike their I/O requirements simultaneously, potentially affecting both network and storage I/O performance. An NVMe flash storage system may be so responsive that even peak loads won't affect performance, but IT planners must take into account even the slightest risk of a performance hit. They should look for quality of service (QoS) capabilities in both the network and the storage systems.
Vendors implement QoS in several ways. Some use a priority-based mechanism to ensure high-priority applications get the majority of I/O capacity when resources become constrained. Others ensure an application always has a specific amount of IOPS available to it at all times. Essentially, the system is thick provisioning I/O capacity. Some systems can set minimum and maximum thresholds to make sure each application never experiences more than a specific level of performance.
Deciding on the best type of QoS depends on the organization. For most, a simple priority based distribution of I/O will be enough, but some may require set guarantees to meet service-level objectives.
Increased parallelism is the primary benefit of NVMe flash storage. That increase means the storage system can support more workloads of varying types. To realize the benefits of NVMe, enterprises must consolidate more workloads on the storage system and increase the density of those workloads with more users per database and more VMs per physical host. The net effect is an NVMe system should drive down the number of physical servers in the environment while expanding the scope of what applications can process.
