Avoid turbulence when shifting to data analytics in the cloud

Adopting an experimental mindset

Companies need to be aware that data analytics in the cloud is not just about cost savings, but also about new possibilities. "With access to scalable, stable infrastructures without the overhead of maintaining them, you can scale to the level of analysis needed for that moment, with minimal startup cost for experimentation," said Mitch Gibbs, a consultant at Candid Partners, a cloud services firm.

Mitch Gibbs

He recommended that analytics managers devote resources to the process of experimentation to determine the analytics approach that offers the most return and to invest in it. "Don't think of analytics as a one-time build of an application," Gibbs advised. "Instead, design your system and process to evolve as the business needs change."

The next step is to set valuable, achievable goals for data analytics in the cloud, such as cutting BI and analytics costs, accelerating queries, boosting user concurrency, improving the quality of decision support and automating delivery of data-driven insights into business processes. "Don't migrate your BI/analytics away from your on-premises platforms if you don't have a good handle on what you're trying to achieve," said James Kobielus, Wikibon lead analyst at SiliconAngle Media.

Don't think of analytics as a one-time build of an application.

There are many SaaS-based BI and analytics tools to consider, and they range widely in features, price, performance, geographic availability, industry and applications. Setting goals can help create a shortlist of providers to target early in the migration initiative. "Perform a due diligence comparative evaluation of these [providers and product features] before deciding which one will be your migration target," Kobielus said. It's important to decide whether your company is just moving operational reporting or if it's also migrating predictive modeling, data mining, machine learning and other advanced analytics applications to the cloud.

Prepare for a migration project that may take longer and cost more than expected, Kobielus noted. The project is likely to be more complex if migrating many databases and a huge collection of analytics that need to be rebuilt essentially from scratch for the cloud.

Here are some important considerations when identifying migration expertise and selecting tooling:

• Are you migrating every legacy BI and analytics app, or are you planning to decommission many of the underutilized ones as part of the migration?
• Do you have the requisite in-house expertise and tooling to do the migration properly, or do you need to bring in a consultant?
• Does the provider of the target cloud have professional services and tools to assist your migration?

Auditing existing data management practices

Rob Lancaster

It's important to assess the data management infrastructure and security around the existing data. "A major issue we see is that the data was traditionally protected by homegrown systems on premises that will not exist in the cloud," said Rob Lancaster, general manager of cloud at Immuta, a data management platform for AI. Organizations sometimes realize too late that once they migrate their data to the cloud, they can't protect it like they used to and need to consider different and more flexible strategies to enable true data analytics.

"We should be mindful of the wreckage of the data warehouses of the past," said Ben Newton, director of product marketing at Sumo Logic, a log management and security analytics company. "There is too often an obsession with just gathering data rather than answering questions."

Sam Boonin

Clearly outline some key business questions and identify the data to answer those questions before migrating data to the cloud. Even better, pick a particular application or business area to start with. "Don't try to boil the data lake. Start with a data pond," Newton advised. He added that, all too often, he has encountered companies building a business strategy on well-nurtured data sets that don't reflect reality. For data analytics in the cloud to be successful, enterprises need to get into the nitty-gritty details for unstructured, structured and semi-structured data analysis. That will make it easier to develop a strategy for addressing the needs of traditional, well-behaved BI tools and the wild west of machine data analytics.

"It's best to start with data that likely already exists in the cloud, such as digital customer journey data or data that is tied to existing investments in SaaS," said Sam Boonin, vice president of product strategy at Zendesk. That will help get some quick wins and build familiarity with the cloud-based BI environments. Then, build a cloud transformation plan into the overall BI strategy and move the rest of the data over time.

Joe Pasqua

Typically, 90% of BI's challenges are accessing, cleansing and normalizing data. The cloud makes these tasks easier since so much data already resides in public clouds like AWS and Microsoft Azure. But Boonin emphasized that a company's "data plumbing" still requires consistent governance and IT work.

Controlling data and costs

A growing concern with data analytics in the cloud, particularly with new regulations like GDPR, is protecting sensitive information during the migration process. Sensitive data needs to be masked or tokenized.

The physical location of data is also an issue. "Since you can't always know or control where the cloud provider stores the data, organizations may inadvertently violate data residency limitations," said Nimrod Vax, co-founder and chief product officer at data privacy company BigID. Companies need to know not only where their data is stored, but also whose data they're storing. Those that can map the data before migration to the cloud will better understand what type of data is being migrated, Vax said.

Many organizations have experienced an 'oh no' moment when a large, unexpected bill arrived.

Cloud pricing can be appealing and seem to be an easy entry point, but costs can be unpredictable. "Many organizations have experienced an 'oh no' moment when a large, unexpected bill arrived," said Joe Pasqua, executive vice president of products at MarkLogic, an operational database management systems provider.

Cost estimation is particularly challenging for BI and data analytics in the cloud. While operational workloads are often driven by a repeatable business process that can make them more predictable, BI and analytics can be very much driven by users and data scientists. "There is always another analysis to do, and the cloud makes it very easy to consume more resources," Pasqua said. "It's very important to use a platform that can effectively analyze usage patterns and control usage in order to attain predictable costs."