Essential Guide

Browse Sections

Editor's note

Storing hundreds of terabytes and even petabytes of data is no longer uncommon for organizations throughout a vast array of industries, but there's a big difference between big data sets composed of stagnant archived files and big data containing business-critical streaming media files. These are commonly referred to as big data at rest, and big data in motion.

When determining what type of storage should house a big data set, it's important to take into account how files will be used. While object storage and scale-out network-attached storage (NAS) are two of the most popular storage options for big data environments, media and entertainment companies that rely on data in motion shouldn’t overlook including higher-performance storage, such as solid-state drives (SSDs).

SearchStorage constructed this guide to help storage pros craft an architecture to house big data sets containing large streaming files. From the links provided throughout this guide, learn about the storage challenges you might encounter with big data and how to work around them, as well as the best way to perform analytics. You'll be well on your way to building the best architecture for your big data in motion.

1Exploring big data storage options

Obvious storage choices for many big data platforms include scale-out NAS, for the amount of capacity it has, and object storage, which can help with the unstructured nature of big data sets. But when streaming files, it can also be a good idea to look at high-performance storage, such as SSDs. The stories and podcasts below include examples of what a storage system should provide for big data, and when specific types of storage work the best in big data environments.

2Choosing storage to accommodate big data analytics

Analytics is one of the most important parts of big data, but it can cause a lag in performance. In addition, the type of storage used can affect analytics efficiency. To make big data analytics effective, storage technologies, such as in-memory data grids, and advances in Hadoop continue to evolve. View the links below to learn more about how storage type affects analytics, and which tools can help you gain the most insight from the contents of big data sets.