Linux Foundation to standardize, simplify the DPU ecosystem

The Linux Foundation has launched the Open Programmable Infrastructure project to standardize the software stack and APIs supporting data processing units to make them easier to use in enterprise data centers.

OPI will define the DPU and develop standardized software frameworks and application programming interfaces (APIs) to make DPUs, also called infrastructure processing units (IPUs), easier to deploy in enterprise data centers, the Linux Foundation said this week.

DPUs are smartNIC semiconductors dedicated to offloading networking and communication functions from the CPU. Businesses pursuing digital transformation are producing more data than ever before. Having dedicated network and security silicon helps to reduce latency in network traffic.

The OPI project will create a community of silicon vendors, device manufacturers, software vendors, test and measurement companies, OEMs and end users to develop standard architectures and frameworks for the software stacks that run on DPUs. Founding members of the community include Dell Technologies, F5, Marvell, Nvidia, Intel and Red Hat.

Cloud providers and hyperscalers use DPUs today to meet the latency and reliability requirements of moving massive amounts of traffic. While hyperscalers code infrastructure to support DPUs from the ground up, enterprises need vendors to provide standardized tools for tailoring DPUs for specific tasks. DPU vendors today provide proprietary software stacks that are not interoperable with other vendors' silicon nor widely supported by device manufacturers.

The project will also develop APIs for connecting the standardized software stacks to other elements in a DPU ecosystem, including hardware, applications hosted in the cloud, and software that provides remote provisioning and orchestration. Simplified and standardized APIs will help ensure applications are portable between a company's own data center and a cloud data center, which might use a different DPU in its infrastructure.

"This brings the industry together over a common standard, [so] you can use DPUs anywhere," said IDC analyst Brandon Hoff. Once a common set of standards have been widely adopted, DPUs "become really off the shelf -- they're ubiquitous, they're easy to use -- and it moves from a niche market to a general market."

Hoff anticipates a widely supported standard in three to five years. Prepackaged DPUs in enterprise hardware will likely take five to 10 years.

The Infrastructure Programmer Development Kit (IPDK), an open source framework of drivers and APIs that the Linux Foundation developed to offload and manage network infrastructure, has been made into a subproject of the OPI initiative. OPI will use the IPDK to build the initial software stacks and drivers.

Nvidia has contributed its open source DOCA software development kit to the OPI project. DOCA includes libraries, drivers, documentation, services, management tools and sample applications.

Other goals of the OPI project are to agree on a standard definition of a DPU, create an ecosystem of open source applications to run on the DPU, and integrate the technology with the Linux kernel. The OPI project plans to foster an open source application ecosystem for the DPU by integrating with open source Linux Foundation projects like the Data Plane Development Kit and Open vSwitch. DPDK provides data plane libraries and drivers for network interface cards. Open vSwitch is a virtual switch designed to enable network automation.  

Ultimately, using DPUs in the enterprise will help network managers run more efficient data centers, proponents said. Modern technologies like 5G, deep learning and cryptocurrency require high-speed network capabilities and packet processing to support the volume of data they produce.

Moving virtualization software onto the DPU will also increase efficiency by allowing more virtual machines to run on a single physical server.

Madelaine Millar is a news writer covering network technology at TechTarget. She has previously written about science and technology for MIT's Lincoln Laboratory and the Khoury College of Computer Sciences, as well as covering community news for Boston Globe Media.