What a big data strategy includes and how to build one

Smart organizations use vast quantities and various types of data to better understand their customers, track inventory, improve logistics and operational processes and make informed business decisions. Successful organizations also understand the importance of managing the burgeoning amounts of big data they are creating, and of finding reliable ways to extract value from them. Having a big data strategy to effectively and efficiently store, manage, process and apply all that data is critical.

A well-defined and comprehensive big data strategy lays out what's needed to become a more data-driven -- and thus successful -- organization. It should incorporate guidelines to help accomplish the data-driven vision and direct the organization to specific business goals for big data applications. This is easier said than done, but the four steps outlined here can help make it doable.

What is big data?

It's not just the size that matters with big data. Data volume is only one of the V's of big data, and managing it is actually one of the easier challenges to solve. The harder challenges of big data have to do with the other V's: the variety of data types, the velocity at which data changes, the veracity of data from different systems and other characteristics that make it difficult to deal with lots of constantly changing data.

Big data can come in many forms, including a combination of unstructured, semistructured and structured data types. It also comes from many different sources, such as streaming data systems, sensors, log files, GPS systems, text, images, audio and video files, social networks and conventional databases. Some of those sources may add or update data up to millions of times a minute.

Not all of the data is created equally. As a result, companies need to ensure that sets of big data from different sources are accurate and trustworthy. This highly variable data might also need to be augmented with additional data from other repositories. For businesses, the ability to handle all these challenging aspects is the key to unlocking the power of big data. That starts with a solid strategy.

The importance of a big data strategy in the enterprise

Too often enterprise data is stored in silos -- dumped in data warehouses or stuck in disparate departmental systems that lack data integration, making it nearly impossible for companies to get a comprehensive view of all their data. Additionally, both data quality in sets of big data and the trustworthiness of data sources can vary, and storage and associated data management costs can be high.

As a result, building a big data strategy is put on the back burner as enterprises scramble to manage and deal with day-to-day business operations. Without a strategy in place, however, enterprises will end up dealing with various big data activities happening simultaneously throughout the organization. This can create duplicate efforts, or worse, competing efforts that are not in alignment or that do not clearly meet the company's long-term strategic objectives.

What should a big data strategy include?

An effective big data strategy provides a clear roadmap for the ways in which the data will be used to support and improve how business is done and the approaches that will be used to manage the big data environment. The plans it incorporates must be actionable, widely adopted and predicated on an enterprise-wide appreciation that data is an asset that sets up the business for continued success. A strategy should also specify how the challenges described above will be addressed.

The key to creating a successful strategy is to not just approach big data as a technology problem. It's important to talk to the business stakeholders and get feedback from them. Doing so will help ensure that the strategy is adopted: Many aspects of big data management have as much to do with cultural alignment as they do with technology enablement. Business managers and senior executives must support the big data strategy and take part in the process.

How to build a big data strategy

Simply investing in big data technologies and tools without an architectural and structural plan can waste time, money and resources in an organization. Here is a four-step approach to formulating a big data strategy that will avoid those negative outcomes.

Step 1. Define the business goals and objectives

It should come as no surprise that in order to have a successful big data strategy, you must first define what business objectives you are trying to accomplish. Not every business is the same, so there is no one-size-fits-all answer here. However, you should make sure that your strategy aligns to your overall corporate business objectives while also addressing key business problems and key performance indicators.

Make sure that stakeholders -- including folks from your data management team, line-of-business leaders, data engineers, data scientists and anyone else who will be utilizing your big data stores -- are involved right from the start and provide key input on a continuous basis.

Step 2. Identify data sources and evaluate processes

The next step involves identifying the variety of your data as well as assessing current business processes, data sources, data assets, technology assets, capabilities and policies at the organization.

Once you have identified sources of data, run an assessment on your data strategy. Make sure to address the business objectives you outlined in step one and work from there. For example, if a business objective of your data strategy is to improve customer experience, then your current state assessment would cover any business processes, business models or data assets that touch customers. When assessing your current state, it's good practice to interview and involve all relevant employees and stakeholders.

Step 3. Identify and prioritize big data use cases

Don't boil the ocean applies here. In formulating a big data strategy, start small, think big, iterate often -- and think in terms of use cases. Identify big data use cases that meet your business objectives outlined in step one. Use big data analytics to examine your large volumes of data to uncover hidden patterns, correlations and other insights. These exercises should help you build out and refine use cases.

The next step is to start prioritizing these use cases based on factors such as their business impact, budget needed and resources required. Depending on how many different departments you have represented in the process, narrowing down use cases and prioritizing which ones to start with may be difficult. Remember to stay focused, write down use cases as they are agreed upon and work as a group to come up with a plan.

Step 4. Create a roadmap for big data projects

Once you have identified your business objectives, gotten an understanding of your data and current capability state and identified use cases, you can now begin to plot out a big data roadmap.

This crucial step is often the most time-intensive step for organizations. When creating your big data roadmap, remember that it's just an outline. You can continue to iterate and evolve your roadmap over time. With that in mind, picture your desired end state and work backward, making sure the end goal is precise, certain and direct.

The roadmap exercise should focus on identifying any gaps you have around data architecture, technologies and tools, processes and skill sets. The gap analysis will likely prompt a review of the use cases prioritized in step three. Again, business stakeholders will play a key role in prioritizing these initiatives based on complexity, budget and cost vs. benefits.

How to ensure your big data strategy is adopted

No strategy is effective without a plan to for ensuring it is actually used throughout your entire organization. Therefore, it's important to consider the following:

• Identify infrastructure challenges. To effectively leverage your data, especially data living in historical databases or antiquated systems, you may need to change IT infrastructure that isn't compatible with big data technology. Identify areas where infrastructure changes will be necessary and involve your stakeholders to ensure different departments and users don't lose access to critical data.
• Evaluate employee resources. It's one thing to create a killer strategy, but without the right roles and skill sets in place, the strategy will go nowhere. Don't underestimate your human resources department in your big data strategy. Your big data team must have the necessary skill sets needed to make sense of data, and then translate those findings to various line-of-business leaders. Without a good team, the entire vision may suffer. HR can play a critical role in finding and hiring the talent you need. Also, don't underestimate the capabilities of current employees. Sometimes with a little reskilling or upskilling your current employees are able to fill critical roles needed to execute your strategy.
• Be agile. You want to make sure that you build flexibility into your roadmap. You need to be able to quickly and easily adjust your budget, employees, use cases and priorities based on the changing circumstances and insights gathered.

Taking the next step in your enterprise big data plan

Staying agile might be the most important principle to live by when implementing a big data strategy. Because data sources and big data technologies are not static, a big data strategy cannot be a one-and-done exercise. But an ongoing, dedicated effort to think strategically about the data will pay dividends. A well-thought-out, well-executed and flexible plan will help your enterprise gain valuable business intelligence, make better business decisions and potentially reshape your business strategy.