NVMe over Fabrics startup Lightbits Labs adds clustering
Lightbits Labs' LightOS now enables application servers to access NVMe flash SSDs in a cluster of servers -- not just a single box -- over standard Ethernet via NVMe over TCP.
Lightbits Labs added high availability to its software-defined block storage in version 2.0, giving it a "real enterprise" NVMe flash product with no single point of failure.
Lightbits Labs' LightOS 2.0 lets users access a pool of NVMe flash drives in a cluster of servers connected via Ethernet networking. The startup's LightOS 1.0 product used TCP-based NVMe over Fabrics (NVMe-oF) to enable multiple application hosts to share NVMe SSDs in a single target storage server. LightOS 2.0 supports clusters of three to 16 storage servers for high availability, so no single node or SSD failure will cause a troublesome service interruption. The new version allows users to scale storage and compute independently.
"They could survive drive failures before, but if the storage target server went down, it was over," said Eric Burgener, a research vice president in IDC's enterprise infrastructure practice. "Now they have a real enterprise product that can be deployed by a much broader group of customers and workloads."
Lightbits Labs targets smaller cloud service providers, companies building private clouds and organizations running latency-sensitive enterprise applications. Use cases include single-instance databases such as MySQL, Postgres and MariaDB, distributed databases such as MongoDB, Cockroach DB and Cassandra, analytics, artificial intelligence and log processing.
Uses NVMe over Fabrics with TCP transport
Burgener said Lightbits Labs differs from many NVMe all-flash array startups in that it requires no client software and supports industry-standard hardware and software, using TCP for the NVMe over Fabrics transport. He said Lightbits Labs' product should be "cheaper and easier to deploy" than NVMe-oF alternatives that use Fibre Channel, RDMA over Converged Ethernet (RoCE) or iWarp.
TCP-based NVMe-oF is slower than RDMA transport options such as RoCE and iWarp, but Burgener said its performance would still provide a boost for customers. He said Lightbits Labs won't be the only vendor to support NVMe/TCP, but unlike Excelero, for example, it doesn't have a host-side software footprint. Hardware vendors Pavilion Data, Infinidat and Pure Storage also support or are preparing to support NVMe over TCP.
Many have been waiting for NVMe/TCP because RoCE doesn't scale very well.
Eric BurgenerResearch vice president, IDC
"Many have been waiting for NVMe/TCP because RoCE doesn't scale very well, " Burgener said. "The real difference here might be that the NVMe/TCP spec was mostly written by engineers at Lightbits. They were first to market with an NVMe/TCP implementation."
Lightbit Labs submitted its NVMe-over-TCP drivers for open source use and relies on the standard drivers built into Linux, including the latest distributions from Red Hat and SUSE and CentOS, Debian, Fedora and Ubuntu. Those running older Linux versions can get NVMe/TCP driver at kernel.org or reach out to Lightbits Labs, according to Josh Goldenhar, the startup's vice president of product marketing.
Goldenhar said Linux-based application servers could attach to logical volumes in the clustered storage servers running LightOS in much the same way they would with an iSCSI array. The logical volumes appear as block devices, and host servers access them as if they were local SSDs, even though requests traverse Ethernet networks via NVMe over TCP, he said. LightOS uses standard NVMe-oF multipathing to enable fast failover if a target goes down and to facilitate non-disruptive software upgrades.
Lightbits Labs' LightOS 2.0 can provide access to NVMe SSDs in a cluster of servers via NVMe over TCP.
Kubernetes CSI support
Additional new capabilities in LightOS 2.0 include a Container Storage Interface (CSI) plugin for Kubernetes to provide persistent storage for stateful containers, and an OpenStack Cinder plugin. Goldenhar said future LightOS releases would add data services such as snapshots and clones.
The LightOS software product runs on standard Linux servers. Lightbits Labs also sells an optional SuperSSD appliance on Dell hardware for customers that prefer preloaded, pre-certified servers. In addition, the startup offers an optional LightField accelerator card to speed inline compression and erasure coding.
Goldenhar said TCP-based NVMe over Fabrics would provide LightOS users with a significant performance boost over iSCSI arrays. He said Lightbits Labs recommends 25-Gb Ethernet for the application servers and 100 Gigabit Ethernet for the target systems, although customers can use slower networking options.
Lightbits Labs designed LightOS for block storage, but the product can also integrate with file systems such as IBM Spectrum Scale, Lustre and BeeGFS, according to Goldenhar. He said Lightbits has also worked on integration projects with open-source MinIO to provide an S3-compatible object store.
List pricing for LightOS ranges from $9,400 to $18,800 per server, depending on the CPU sockets and/or network cards, according to Goldenhar. He said about 60% of Lightbits Labs' customers go the software-only route, and the other 40% use the SuperSSD appliance and/or LightField accelerator card.
Goldenhar said Lightbits Lab has 10 to 50 customers, with about half paying and half in the proof-of-concept stage. He cited examples of a New York-based hedge fund that is running Apache Kafka to do real-time fraud protection and two financial trading firms that are working on algorithmic trading verification.
