Predicting data storage growth

By Yuval Shavit, Features Writer

As a company grows, the amount of data it uses also increases. While that comes as no surprise, most companies don't have a plan in place to address the problem of storing all that information. While it's relatively easy to see how much of a client's disk space is currently being used -- you just go in and look at the servers -- measuring how quickly data is growing isn't as trivial. In the last installment of our Hot Spot Tutorial on storage capacity planning, we looked at how accurate planning can save your clients money. In this article, we'll show you how to make predictions about data storage growth.

The easiest but most expensive way to measure data storage growth is to use software designed specifically for storage capacity planning. Several vendors develop storage resource management (SRM) tools, and most of those tools are more or less the same, said Henry Baltazar, storage analyst at The 451 Group in San Francisco. SRM software tracks how full your client's disks are and the rate at which data is growing. The software can create detailed reports and recommend when your client will need a hardware upgrade.

The downside to SRM tools is that they are designed for larger companies and are expensive. Since the primary driver for storage capacity planning is to save money on hardware, recommending a software solution that costs your client more money than it'll save on hardware doesn't make sense, said Kurtis Lindemann, director of technical services at RoundTower Technologies, a storage consultancy in Cincinnati. For clients with fewer than about 10 servers, a more manual approach usually makes sense, Baltazar said.

Measuring and projecting data storage growth

The best way to track growth is to use an SRM tool over several months. If that approach is too expensive, you can manually estimate your client's past rate of data growth to predict future growth. Companies are usually happy to have you perform this initial estimate, Lindemann said, as long as it's free -- it's hard to convince them to pay for an assessment of something they may not yet see as a major problem. One of the goals of this assessment is to determine how accurate your client's own predictions about data storage growth have been. If you can demonstrate that your client's estimations have been off, you can sell them on the money they'll save by being able to plan storage hardware purchases better.

One way to measure growth manually is to scan your client's disk and look at the files' metadata, like creation and modification dates, to build a past history. While this method gives you a good estimation, it doesn't take into account deleted files, Lindemann said. Such an omission makes it appear that there was less data at a certain date than there actually was and thus inflates the rate of growth since that date.

Another option is to look at old backups and use their rate of growth as an approximation for the overall rate of growth within the company. If you do this, it's important to understand what the company's backup policies are, and how they've changed. For instance, if your customer started backing up employees' drives a year ago, that would show up as a surge in the volume of data backed up. Even though the company had about as much data overall before and after the policy change, it would appear as if the data's rate of growth suddenly jumped.

Whether you use SRM tools or a manual approach, you should work with the client to track data storage growth going forward. For many smaller companies, the best storage capacity planning software is a spreadsheet, Baltazar said. Especially since the non-SRM tools can only give estimates of storage capacity and data growth, an ongoing process can help your client refine those estimates and compare the predictions with real results.

If scanning existing files' metadata or looking at backup tapes prove to give a good enough estimate, you can pitch those methods as low-end SRM replacements. Lindemann said several customers ended up buying a copy of the disk scanning software he used during the initial assessment he did for them.

Preparing for future growth

Once you have a handle on your client's current capacity usage and how fast data is growing, you can start implementing a plan for accommodating future storage capacity. As you come up with that plan, you should take into account not just the rate of data storage growth, but also the overall capacity and how much of it is being used. A company that uses 80% may be in trouble, if the overall capacity is small and the rate of growth is relatively large; it could also be safe, if the overall capacity is large and the rate of growth is small, Baltazar said. You should also consider how often your client wants to upgrade its storage capacity; the longer between upgrades, the more capacity each one will have to add.

The goal of storage capacity planning is to fit storage purchases as closely as possible to the function of storage capacity requirements, so it's better to have more frequent purchases of smaller amounts. But if your client needs to do a data migration each time it adds a storage device, the cost of each upgrade goes up dramatically. If your client is large enough to afford it, you may want to recommend a storage area network (SAN) that supports easy expansions. Most SANs have this ability, which lets you simply buy another storage module and attach it to increase the SAN's capacity without having to migrate all of its data.

If your client can't afford a SAN, you'll have to help come up with a compromise between buying too much storage at once and having to do too many costly upgrades. The good news is that data migrations are vital services that you can offer, Baltazar said. It's important to get a data migration right, so data isn't lost, but it isn't very fun. Companies are often willing to pay a reseller or consultant to take the job from their hands and do it right, he said.

Being able to predict data storage growth is a crucial part of storage capacity planning, but it's also important to try to curb the rate of growth itself. The more you can reduce how fast your client's data grows, and the more you can manage what growth there is, the more your client will save. In the final installment of our Hot Spot Tutorial on storage capacity planning, we'll look at some of the best practices of storage capacity planning and how to complement your predictions with other ways to save your client money on storage.