software-defined storage (SDS)

What is software-defined storage (SDS)?

Software-defined storage (SDS) is a software program that manages data storage resources and functionality with no dependencies on the underlying physical storage hardware.

Purists argue that any data storage product could be described as software-defined since all storage products require software to manage the underlying hardware and control storage-related tasks. However, the marketing term software-defined storage is most often associated with software products designed to run on commodity server hardware with Intel x86 processors. These programs enable cost savings compared with traditional storage area network (SAN) and network-attached storage (NAS) systems that tightly couple software and hardware.

Why is SDS important?

Unlike monolithic SAN and NAS systems, software-defined storage systems let users upgrade the software separately from the hardware. Common characteristics of SDS products include the following capabilities:

• Aggregate cost-effective storage resources.
• Scale out the system across a server cluster.
• Manage the shared storage pool and storage services through a single administrative interface.
• Set policies to control storage features and functionality.

Factors contributing to an increase in the number of SDS products include the following:

• The explosive growth of unstructured data, creating a greater need for scale-out storage architecture.
• The availability of high-performance server hardware with multicore processors.
• The general acceptance of virtualization in servers, desktops, applications and networking.
• The growing popularity of cloud environments.

Use cases for software-defined storage

SDS use varies by product type. SDS can accommodate block, file and object storage applications. Common use cases for scale-out object and file SDS include apps that generate significant amounts of unstructured data, such as data analytics, genomics and internet of things (IoT).

Scale-out block SDS often targets higher-performance workloads, such as databases. Many types of SDS are used in DevOps environments that require flexible storage provisioning for new applications.

SDS is part of an ongoing industry trend that also includes software-defined networking (SDN), software-defined infrastructure and software-defined data centers.

Types of SDS products

SDS can be difficult to categorize because of the lack of an official standard definition. While SDS products typically support block, file and object storage interfaces, vendors tend to prioritize one or two interfaces. Others SDS products are accessible through one or two storage protocols. Some SDS products that started as object storage have added support of file protocols, and some distributed file systems also support data management capabilities that offload data to object storage.

Many SDS products run on the server operating system (OS) and in a virtual machine (VM), whether on premises or in a public cloud. Other SDS products run only in a server hypervisor kernel or VM. Some SDS products run in a container. This approach conserves server resources and facilitates the consistent management of container-based applications and storage services through a single container orchestration tool.

SDS providers generally provide lists of certified hardware options. Some SDS vendors sell products that package software with standard server hardware to ease procurement and deployment for customers. Many SDS products provide scalability features that let users scale compute and storage resources separately. A concern here is that users may find themselves locked into a particular vendor.

Hyperconverged options scale storage, compute, virtualization and networking in the same physical hardware. Hewlett Packard Enterprise and Nutanix are among the vendors selling hyperconverged infrastructure software packaged with standard hardware.

Leading storage vendors released software versions of storage products that were previously tied to specific hardware. Examples include the following:

• Dell EMC UnityVSA from its Unity storage array and IsilonSD Edge from the Isilon scale-out NAS system.
• IBM Spectrum Accelerate from the vendor's XIV storage.
• NetApp OnTap Select, a software-only version of the OS that powers the company's storage arrays.

Open source SDS is freely available through community development projects. Examples of open source SDS include Red Hat Ceph, TrueNAS (formerly FreeNAS), MooseFS and OpenStack Swift. Commercially supported distributions of open source SDS are available from various vendors.

Advantages and limitations of software-defined storage

SDS has pros and cons. Among the advantages, SDS can pool storage and swap storage devices in and out, independent of the storage vendor.

SDS also has better administrative and capacity management than traditional storage. Large storage pools can be created that are visible to a variety of devices and applications. Users can select the highest-quality storage devices, while still using legacy storage infrastructure for specific requirements. In situations where additional storage is needed quickly, SDS can burst storage capacity using cloud-based storage. It can handle all types of file systems.

Conversely, SDS faces bottlenecks when working with storage devices that have unique or highly specialized capabilities. This can require using a vendor-supplied dashboard to manage that vendor's storage attributes.

Some vendors package their own SDS with one of their storage devices, making it difficult to use the SDS resource effectively. Despite SDS benefits, users might still want to deploy a hardware-based storage array.

SDS vs. storage virtualization

In contrast to SDS, storage virtualization is a technique for managing storage in which the entire array of available block storage and file storage is abstracted into a virtual pool of storage. Storage resources are mapped to make it easier to locate stored data.

Both SDS and storage virtualization can effectively manage storage devices from different vendors. They also both provide resources for business continuity and disaster recovery and have the ability to quickly scale resources as needed. As with any storage solution, business requirements are a key aspect of the decision-making process.

SDS planning guidelines

Follow these steps when designing and implementing an SDS solution, whether for a new storage arrangement or an existing one:

1. Understand business needs for data storage security, accessibility and protection.
2. Check that senior management supports the initiative.
3. Apply the systems development lifecycle as a useful model for evaluation, selection and implementation.
4. Review storage policies, procedures and protocols to see if they're still relevant or need updating.
5. Identify and analyze on-premises and cloud storage options to see if they fit the current storage environment.
6. Determine how a change in storage will impact operations, costs and other factors using cost-benefit analysis.
7. Select a specific approach, and get all necessary approvals.
8. Prepare a project plan, and divide it into multiple phases.
9. Once the project is completed, test and validate it.
10. Update all relevant storage policies, procedures and protocols.
11. Provide training for IT storage teams and employees on the new systems.
12. Periodically assess the new storage deployment, validating the benefits and briefing senior management on results.

The future of SDS

Several trends are likely to impact the continued acceptance and use of SDS:

• AI-driven intelligence. SDS vendors are building artificial intelligence into their systems to optimize storage management and performance by predicting workload patterns, dynamically allocating resources and boosting efficiency.
• Convergence with other software-defined technologies. SDS is steadily converging with other software-defined technologies, such as SDN and software-defined computing.
• Containerization. SDS solutions are adapting to containerized environments, providing persistent storage for containers.
• Edge computing. SDS will provide scalable, reliable storage solutions for edge computing sensors and IoT devices.
• Solid-state drives. SSDs are an essential technology for SDS; they excel in high-performance requirements, managing large amounts of data with low latency.

Find out more about software-defined storage products available from the big storage players, software-defined storage specialists and SDS options focused on public and private clouds, virtualization and containers.