New NVMe interface specification focuses on management
An NVMe management interface builds on the existing standard, allowing users to monitor NVMe devices. NVMe-MI may represent a significant step forward for the technology.
The NVMe interface is quickly becoming the industry standard for establishing communications between in-band host software and SSDs. More recently, the NVMe Management Interface specification has been added as an adjunct to NVMe to provide support for out-of-band management. The NVMe-MI standard defines a command set and architecture for controlling NVMe storage, making it possible to discover, monitor and update NVMe devices using a remote management application.
NVMe is an interface and command set for communicating with storage that attaches to a computer via a Peripheral Component Interconnect Express (PCIe) serial expansion bus.
PCIe is a bus standard commonly used for connecting peripheral devices to a computer, providing lower latency and higher data transfer rates than its predecessors. Each peripheral device that attaches to the motherboard via a PCIe bus has its own point-to-point connection. In this way, multiple PCIe devices attached to the same computer are not competing for bandwidth.
The NVMe interface addresses many of the limitations of earlier storage technologies that were unable to take full advantage of PCIe SSDs, offering improvements in IOPS, bandwidth and latency. As a result, NVMe is quickly becoming the de facto standard for these PCIe drives.
NVMe facilitates communications between host software and PCIe non-volatile memory subsystems. An NVM subsystem is a PCIe storage module that includes the components needed to carry out NVMe communications. In addition to the storage media, the subsystem contains one or more NVMe controllers, zero or more PCIe ports and several other components.
Introducing NVMe-MI
NVMe-MI builds on the NVMe interface specification to provide an infrastructure for managing NVMe devices. NVMe-MI is an architecture and command set that enables direct out-of-band administration of NVM subsystems. The NVMe-MI command set defines the command messages that the management controller is permitted to submit. The set includes NVMe Admin commands, commands specific to NVMe-MI and commands that provide access to PCIe configurations.
According to the NVMe-MI specification, NVMe-MI supports several key operations:
discovering NVMe devices and their capabilities;
storing data about the host environment and making it available to the management controller;
In addition to these operations, NVMe-MI provides multiple command slots to prevent long latency commands from blocking monitoring operations. NVMe-MI also supports the standard Vital Product Data format and provides the ability to read and write VPD content.
Most importantly, this NVMe interface specification is system-agnostic. It makes no recommendations or assumptions about NVM storage, system processors or operating systems, nor does it suggest any usage models. For example, the specification does not state whether NVMe should be used with SSDs, main memory, cache or other types of memory. The specification's primary goal is to provide a common management interface at a level that abstracts the implementation details.
The NVMe-MI protocol layers
NVMe-MI is an architecture and command set that enables direct out-of-band administration of NVM subsystems.
An NVMe-MI infrastructure is made up of an application layer, protocol layer, message transport layer and physical layer that work together to carry out management-related communications:
The application layer includes two components. The first is the management application used to administer the NVM subsystem. The second is the management controller that communicates with the subsystem on behalf of the application.
The protocol layer consists of the actual NVMe-MI interface associated with each management controller. The interface executes the management operations against the NVM subsystem, working with the subsystem's management endpoints.
The message transport layer is made up of the Management Component Transport Protocol and the MCTP bindings used to interface with the physical layer. MCTP is a protocol that supports management-related communications between intelligent hardware devices. NVMe-MI utilizes MCTP to facilitate communications between the management controller and management endpoints. The message transport layer supports both PCIe and System Management Bus-I2C bindings. SMBus and I2C are compatible two-wire buses.
The physical layer includes the ports necessary to receive messages from the message transport layer, with each port associated with one management endpoint. The physical layer can include zero or more PCIe ports, as well as zero or one SMBus-I2C ports.
All NVMe-MI communications occur between the management controller in the application layer and the management endpoints in the physical layer, passing through the protocol layer and message transport layer en route. In other words, all NVMe-MI communications originate or terminate with the management controller or management endpoints.
The management endpoints associated with the PCIe and SMBus-I2C ports all support the same NVMe-MI commands and provide the same functionality, including the ability to advertise their capabilities. However, PCIe ports support much higher data rates than SMBus-I2C ports.
NVMe-MI keeps getting better
The latest revision to the NVMe-MI specification is version 1.1, which adds the ability to tunnel NVMe commands over an NVMe Admin queue. Because NVMe over Fabrics also supports the NVMe Admin queue, NVMe-MI commands can now be sent over an NVMe-oF connection. In addition, NVMe-MI 1.1 also adds support for in-band management, as well as the ability to manage SSD enclosures, including elements such as fans and temperature sensors.
Despite the improvements that come with version 1.1, this NVMe interface standard is still a relatively young technology, and it will take time before it becomes widely implemented. Even so, NVMe-MI represents an important addition to the NVMe specification and a critical step toward more effective management of NVMe devices. NVMe-MI also points to the important gains that NVMe has made recent years and how essential it has become in supporting SSD devices. With such widespread adoption, it's just a matter of time before NVMe-MI also becomes a regular part of the IT infrastructure.
