Hewlett Packard Enterprise subsidiary Cray delivers new ClusterStor E1000 all-flash and disk storage arrays, with hardened Lustre file system and ClusterStor Data Services.
Supercomputer maker Cray has pumped out revamped high-density ClusterStor storage, its first significant product advance since being acquired by Hewlett Packard Enterprise.
The new Cray ClusterStor E1000 launched this week, six months after HPE's $1.3 billion acquisition of Cray in May. Engineering of the E1000 began before the HPE acquisition.
Data centers can mix the dense Cray ClusterStor E1000 all-flash and disk arrays to build ultrafast "exascale" storage clusters that converge processing for AI, modeling and simulation and similar data sets, said Ulrich Plechschmidt, Cray lead manager.
The E1000 arrays run a hardened version of the Lustre open source parallel file system. The all-flash E1000 provides 4.5 TB of raw storage per SSD rack, with expansion shelves that add up to 4.6 TB. The all-flash model system potentially delivers up to 1.6 TB of throughout per second and 50 million IOPS per SSD rack, while an HDD rack is rated at 120 Gbps and 10 PB of raw capacity.
When fully built out, Plechschmidt said ClusterStor can scale to 700 PB of usable capacity in a single system, with throughput up to 10 PB per second.
Cray software stack
Cray ClusterStor disk arrays pool flash and disk within the same file system. ClusterStor E1000 includes Cray-designed PCIe 4.0 storage servers that serve data from NVMe SSDs and spinning disk. Cray's new Slingshot 200 Gbps interconnect top-of-rack switches manage storage traffic.
The most impressive work Cray did is on the software side. You might have to stage data in 20 different containers at the same time, each one outfitted differently. ... That's a very difficult orchestration process.
Steve ConwayCOO and senior research vice president, Hyperion Research
Newly introduced ClusterStor Data Services manage orchestration and data tiering, which initially will be available as scripted tiering for manually invoking Lustre software commands. Automated data movement and read-back/write-through caching are on HPE's Cray roadmap.
While ClusterStor E100 hardware has massive density and low-latency throughout, Cray invested significantly in upgrading its software stack, said Steve Conway, COO and senior research vice president at Hyperion Research, based in St. Paul, Minn.
"To me, the most impressive work Cray did is on the software side. You might have to stage data in 20 different containers at the same time, each one outfitted differently. And you have to supply the right data at the right time and might have to solve the whole problem in milliseconds. That's a very difficult orchestration process," Conway said.
The ClusterStor odyssey
HPE is the latest in a string of vendors to take ownership of ClusterStor. Seagate Technology acquired original ClusterStor developer Xyratex in 2013, then in 2017 sold ClusterStor to Cray, which had been a Seagate OEM partner.
HPE-owned Cray released new all-flash and disk ClusterStor arrays for AI, containerized workloads.
HPE leads the high-performance computing (HPC) market in overall revenue, but it has not had a strong presence in the high end of the supercomputing market. Buying Cray allows HPE to sell more storage for exascale computing, which represents a thousandfold increase above petabyte-scale processing computing power. These high-powered exascale systems are priced beyond the budgets of most commercial enterprises.
Cray's Shasta architecture underpins three large supercomputing sites at federal research labs: Argonne National Laboratory in Lemont, Ill.;. Lawrence Livermore National Laboratory in Livermore, Calif.; and Oak Ridge National Laboratory in Oak Ridge, Tenn.
Conway said Cray and other HPC competitors are under pressure to expand to address newer abstraction methods for processing data, including AI, container storage and microservices architecture.
"You used to think of supercomputers as a single-purpose steak knife. Now they have to be a multipurpose Swiss Army knife. The newest generation of supercomputers are all about containerization and orchestration of data on premises," Conway said. "They have to be much more heterogeneous in what they do, and the storage has to follow suit."
