Purchasing an enterprise SAN: Top-tier products compared

Purchasing criteria for high-end SANs

We'll step through a list of purchasing criteria for high-end enterprise SAN arrays and compare these eight top-tier products (information is current as of October 13, 2015):

• EMC VMAX3
• Fujitsu Eternus DX8700 S3/DX8900 S3
• Hitachi Data Systems (HDS) Virtual Storage Platform (VSP) G1000
• Hewlett-Packard (HP) XP7 Storage system
• HP 3PAR StoreServ 20000 Storage family
• Huawei OceanStor 18000
• IBM DS8870 (IBM has rolled out the DS8886, which will replace this system next year)
• NetApp FAS8000 

The following criteria are used as the basis of our product comparisons:

• Hardware architecture
• Disk subsystem
• Performance
• Protocol support and available ports
• Availability and reliability
• Data efficiency features
• Storage management, data protection and integration

Hardware architecture

Scalability is one of the hallmarks of high-end enterprise SAN arrays, so hardware architectures need to support substantial scalability. With the exception of the Fujitsu Eternus DX8700 S3/DX8900 S3, HP 3PAR StoreServ 20000 Storage family, IBM DS8870 and NetApp FAS8000, these products are based on a monolithic scale-out architecture that scales from one to eight active-active controllers that access a shared pool of storage.

More recently released products, such as the EMC VMAX3, HDS VSP G1000, HP XP7 and NetApp FAS8000, scale to multiple hundreds of cores. The IBM DS8870 leverages a symmetric multiprocessing (SMP) architecture of an active-active pair of Power7+ servers. The NetApp FAS8000 is the only product that scales horizontally by adding active-active high-availability (HA) pairs that are loosely coupled; each HA pair is an independent storage system with its own processors, storage and ports. For the very high-end crop of enterprise SAN arrays, the monolithic scale-out approach has proven to scale performance to multiple millions of IOPS and sub-millisecond latency; that, combined with the support of hundreds of cores, gives the architectures of the HDS VSP G1000, HP XP7, Huawei OceanStor 18000 and EMC VMAX3 an edge.

Disk subsystem

Notably, the NetApp FAS8080 EX can support up to 17,800 drives and more than 100 petabytes of raw storage, which makes it the clear capacity scalability leader by an order of magnitude beyond the rest of our enterprise SAN field.

All systems scale to multiple petabytes of raw storage. Notably, the NetApp FAS8080 EX can support up to 17,800 drives and more than 100 petabytes of raw storage, which makes it the clear capacity scalability leader by an order of magnitude beyond the rest of our enterprise SAN field. The ability to virtualize third-party storage, and add it to the shared storage pool while still being able to use all array features, has been a hallmark of both the HDS VSP G1000 and HP XP7. Likewise, the EMC VMAX3 supports virtualization of third-party storage via Federated Tiered Storage (FTS), and NetApp's FlexArray storage virtualization allows its FAS8000 to pool storage from NetApp E-Series, EMC, HP and Hitachi storage systems.

All products support a combination of solid-state storage and HDDs. The IBM DS8870 combines HDDs with 1U high-performance flash enclosures that house 400 GB encryption-capable flash cards with a maximum of 240 flash cards. Besides HDDs and SSDs, NetApp features a combination of read-only PCI Express (PCIe) Flash Cache and a SSD Cache Flash pool, enabling the FAS8080 EX to support up to 144 TB of cache, the largest supported cache of the enterprise SAN products in this comparison. The HP 3PAR StoreServ 20000 Storage family supports up to 1.8 TB of DRAM cache and 32 TB of flash cache. Fujitsu's Eternus DX8900 S3 currently supports a DRAM cache plus up to 67.2 TB of flash cache in addition to HDD and SSD drives.

Most of the products offer AST among multiple tiers of storage, especially between HDDs and solid-state storage, but there are differences in implementation and data movement granularity. Storage tiering extends the capability of cache by keeping active data on fast tiers and less active data on slower tiers. Although NetApp supports substituting HDDs with SSDs for additional solid-state tiers, it has no automated storage tiering for those tiers. NetApp relies on its large NAND flash cache (the combination of Flash Cache and the Cache Flash pool) to keep active data in fast flash storage. 

With increased security threads and the likely use of these enterprise SAN products in cloud environments, encryption of data at rest is important. All products mentioned provide back-end encryption by the array except for the HP 3PAR StoreServ 20000 Storage family, IBM DS8870 and NetApp FAS8000, which all rely on self-encrypting disk drives, and Huawei's OceanStor 18000, which doesn't support encryption.

The NetApp FAS8080 EX is the raw capacity and cache scalability leader. By exclusively relying on a large cache for keeping active data in the fastest tier, it avoids the added complexity of AST but doesn't allow customers to automate data movement of storage tiers beyond cache.

Performance

Although the performance of a SAN array is difficult to gauge as a result of workload dependency and array configuration, a comparison of high-end enterprise SAN arrays wouldn't be complete without discussing IOPS/sec, latency and throughput. We use a combination of Storage Performance Council (SPC) benchmarks, which are array configuration-specific and only available for some products, as well as other information, such as architecture and scalability, to assess performance. The SPC-1 and SPC-2 performance leaders are the HDS VSP G1000 and the white-labeled version HP is offering as its XP7 model; these arrays boast 2 million IOPS/sec (at a $1/SPC-1 IOPS), sub-millisecond latency and high throughput.

Considering its hardware architecture, the 16 TB maximum cache size of the EMC VMAX3 may be able to challenge these numbers, but independent benchmarks were not available at press time. NetApp only submitted a midrange FAS8040 to the Storage Performance Council, so the results are relatively modest. The NetApp FAS family of products runs block-level protocols on top of its Write Anywhere File Layout (WAFL) file system and is based on a loosely coupled scale-out architecture of HA pairs, which is common in scale-out NAS, but uncommon in the very high-end SAN array market.

Protocol support and available ports

All products support 8 Gbps and 16 Gbps Fibre Channel (FC) protocols. Fibre Channel over Ethernet (FCoE) is supported by all products except for the EMC VMAX3 and IBM DS8870. FICON mainframe support, which has been a hallmark of very high-end arrays, is only supported by the HDS VSP G1000, HP XP7 and IBM DS8870 arrays. Native file-system protocol support is available in the EMC VMAX3, HDS VSP G1000, HP 3PAR StoreServ 20000 Storage family and NetApp FAS8000. The Fujitsu Eternus DX8700 S3/DX8900 S3 supports 16 Gbps FC, 10 Gbps iSCSI and 10 Gbps FCoe. The HP 3PAR StoreServ 20000 Storage family and NetApp FAS8000 support the iSCSI protocol, which gives them an advantage in environments that don't have an FC network. Overall, the HDS VSP G1000 and HP XP7 have a protocol support edge in the very high-end enterprise SAN array space by supporting FC, FICON and file-system protocols; the lack of iSCSI isn't a major issue because FC is still predominant for the use cases for which these arrays are usually deployed. Customers who want to run an Ethernet-based SAN can use FCoE.

All the profiled arrays can scale to at least 128 front-end ports, with the EMC VMAX3, HDS VSP G1000, HP XP7, HP 3PAR StoreServ 20000 Storage family and Huawei OceanStor 18000 able to scale beyond 200 front-end ports.

Availability and reliability

One of the main reasons for getting a very high-end SAN array is to ensure undisrupted storage services. Five nines of availability, which relates to a mere five minutes of downtime a year, is the minimum requirement for this class of array, but six nines is the norm. NetApp's FAS8080 meets the five nines minimum, but NetApp points out that there are options to get to six nines by following NetApp best practices. The HDS VSP G1000 and HP XP7 claim seven nines of availability, giving them the highest availability rating among the products discussed in this feature. The EMC VMAX3 will also deliver seven nines of availability if used in conjunction with EMC's VPLEX HA product. All products provide single- and multi-site HA options, and support non-disruptive software upgrades and hardware replacements. End-to-end data protection via the T10-DIF data integrity feature is quite common in very high-end SAN arrays, and is supported by all products with the exception of the FAS8000. This feature helps maintain true end-to-end consistency checking from the storage array through the SAN to the server and back.

Data efficiency features

One of the primary use cases of a high-end enterprise SAN array is storage consolidation of massive amounts of data, so any feature that can save physical storage can provide significant cost savings. All the profiled products support thin provisioning, the ability to provision more storage than is physically present. Similarly, efficient clones and snapshots that take up very little space are common features of high-end enterprise SAN arrays. On the other hand, inline data deduplication of the SSD tier is only supported by the HP 3PAR StoreServ 20000 Storage family, but both the IBM DS8870 and NetApp FAS8000 support post-process deduplication.

Although all the products support QoS, they differ in how QoS is assigned. Most manage service-level objectives by volume; however, the NetApp FAS8080 permits setting QoS policies at volume, file and virtual machine levels. The support of inline encryption of data in the expensive SSD tier gives the HP 3PAR StoreServ 20000 Storage family the edge in data efficiency; being able to maximize the use of the expensive SSD tier results in substantial cost savings.

Storage management, data protection and integration

All products provide a GUI-based management application and command line interface support. Synchronous and asynchronous replication support is indispensable in high-end enterprise SAN arrays, and all the roundup products support both types of replication. From an integration perspective, integrations with VMware APIs are most prevalent and VMware's vStorage APIs for Array Integration (VAAI) and vStorage APIs for Storage Awareness (VASA) are supported by all products.

Bottom line on Big Iron

The HDS VSP G1000 and the HP XP7 version of the array have a slight advantage over the other enterprise SAN products explored in this feature. These products push scalability limits, have the leading SPC performance benchmark scores, boast seven nines of availability and, with the exception of missing inline deduplication, have the features one would expect from a high-end enterprise SAN array, including mainframe support and the ability to virtualize third-party storage. EMC's VMAX3 is a close challenger, but lacks features like FCoE and FICON support, and doesn't have independent benchmark numbers. The IBM DS8870 is an efficient feature-rich, high-end enterprise SAN array that's trusted to run the most mission-critical applications. The HP 3PAR StoreServ 20000 Storage family has clearly joined the top tier of SAN arrays with efficiency features, including inline deduplication. The Huawei OceanStor 18000 is an impressive first-generation product, but Huawei has little global brand awareness in the storage market and the pool of available data storage administrators is small. Similarly, Fujitsu has a limited installed base in the U.S. The NetApp FAS8000 family is at the top when it comes to scalability, storage features and integration; it's also the best NAS product on the market and a great storage option for VMware environments, but it may not be appropriate for very demanding transactional applications.