All-flash Vast Data storage trains AI data at LightSpeed

Vast Data's Universal Storage platform is still expanding.

A new Vast Data storage system aims to accelerate AI projects with LightSpeed. The all-flash startup added NAS capabilities in the first hardware upgrade to its Universal Storage platform since its early 2019 launch. The new product, Vast Data LightSpeed, supports scale-out storage for ultrafast random data access. In a twist, Vast Data positions LightSpeed as a tool for high-performance computing jobs, which typically run sequential workloads on a standard file system.

Vast Data LightSpeed includes a 2U NVMe-enabled enclosure that incorporates Nvidia's forthcoming GPUDirect Storage, which is in beta. Nvidia GPUDirect Storage is designed to reduce latency by writing data from storage to Nvidia supercomputers. The package includes Vast Data algorithms to maximize the efficiency of flash.

Customers can buy LightSpeed clusters in two-node, five-node and 10-node configurations. The NAS component is included as a no-cost enhancement to customers with support contracts.

No-caching architecture based on containers

Vast Data Universal Storage serves all data from flash. Its "shared everything" architecture uses lower cost QLC SSDs for bulk storage with more expensive Intel Optane storage class memory acceleration, connected via NVMe over Fabrics in the box. File and object protocols are presented as container-based microservices. Vast claims its method ensures data remains highly available on the fastest storage media.

The Vast Data storage targets applications that hinge on fast random access, said Eric Burgener, a research vice president of storage at IT analyst firm IDC.

"It's cacheless and has a relatively large memory tier based on storage class memory [compared to] cache-based architectures. All data is accessible to all servers with a single low-latency hop over an RDMA over Converged Ethernet [RoCE] network. Vast now is introducing a NAS capability that makes the software better suited to random workloads," Burgener said.

Companies have to weigh the cost of using flash to train AI models. That's the market Vast Data is trying to capture, but it faces competition from the likes of Dell EMC Isilon, Pure Storage FlashBlade and Qumulo, which sells scale-out storage software directly and on Hewlett Packard Enterprise servers.

Vast Data storage: No disk allowed

Dr. Bruce Rosen's IT team just started unpacking its Vast Data LightSpeed storage system this week. Rosen is director of Martino's Center for Biomedical Imaging, affiliated with Massachusetts General Hospital. The center's sophisticated imaging gear is used by clinical researchers from other institutions as well. Rosen said Martino's needed a fast way to capture and utilize the growing pile of bits of data related to medical-imaging scans.

Dr. Bruce Rosen

"Given the nature of what we do, we are constantly acquiring new image data of the body. We need to figure out a way to use data that in the past would have been discarded. Running the AI model is fairly straightforward, but training it takes a lot of data and lots of time," Rosen said.

Martino's plans to use Vast Data's storage coupled with four Nvidia DGX-A100 AI servers to migrate nearly 2 TB of clinical images stored on a disk-based picture archiving and communication system. Rosen said he will keep the disk storage as a deep archive.

Vast Data has made a splash since its inception with its use of NVMe and QLC in the same system. The company in August added former long-time NetApp president Thomas Mendoza to its board. Venture investment is $180 million, including a $100 million funding round in April that pushed its valuation to $1.2 billion. Vast claims it sold more than 50 PB of data center flash to countries and generated revenue growth of nearly 500% year over year, although the private company does not disclose sales figures.