AWS unifies analytics and AI development in SageMaker

Unification

Data management, analytics and AI are becoming increasingly intertwined.

Historically, data management amounted to storing data in databases and data warehouses until it was needed to inform analytics tools and machine learning models. Meanwhile, analytics meant examining reports and dashboards populated with historical data to discover trends, while AI used that data for applications such as forecasting and scenario planning.

Data and AI are inseparable. ... Anything vendors can do to streamline data and AI workflows will benefit users.

Now, AI and analytics are blending with many generative AI assistants, enabling data analysis and insight generation through natural language interfaces, while data management fuels analytics- and AI-driven analysis with information delivered through automated pipelines.

"Data and AI are inseparable," Henschen said. "Data and AI professionals get familiar with tools of choice, and they create [preferred] workflows, but those processes can be cumbersome. ... Anything vendors can do to streamline data and AI workflows will benefit users."

Similarly, David Menninger, an analyst at ISG's Ventana Research, said data and AI can't be separated, with AI tools requiring large amounts of high-quality data to be accurate. However, due to the separation of data in one environment and development in others, organizations have struggled to put AI tools into production.

"If you can bring more of the information architecture components together, it reduces the complexity and the effort required to create and maintain systems," Menninger said.

AWS provides a broad array of data management, analytics and AI capabilities, he noted. However, before the SageMaker update, they were loosely integrated. As a result, SageMaker and the general availability of Unified Studio are meaningful for users.

"One of the knocks against AWS was that it required a significant amount of knowledge and effort to tie separate pieces together," Menninger said. "Bringing data, analytics and AI together ... will make it dramatically easier for enterprises to use these products."

While SageMaker Lakehouse and SageMaker Catalog provide data management within SageMaker, Unified Studio is its environment for AI and analytics development. Unified Studio includes data processing, SQL-based analytics, machine learning model development and generative AI development capabilities.

In addition, Unified Studio includes select capabilities from Amazon Bedrock, the tech giant's main AI development environment.

Sirish Chandrasekaran, vice president of analytics at AWS, noted that the way competing enterprises can differentiate from one another is with data. All have access to the same generative AI models, development tools and data management platforms. None have the same data.

So, bringing unique data together with the capabilities it informs and making it as simple as possible to do so is important.

"The next generation of SageMaker is about bringing things together," Chandrasekaran said.

Data informs various tools used by various personas within organizations, he continued. Unlike in the past, those tools and personas now interact.

For example, an enterprise might want a customer segmentation table that gets fed into a machine learning model, which in turn goes into an application, which produces new data that feeds back into the customer segmentation table.

"When you think about these workflows and formerly distinct jobs, you have teams coming together that want to work on the same data sets, share assets and move fast," Chandrasekaran said. "Unified Studio brings together all these capabilities."

SageMaker is now AWS' primary platform for data management, analytics and AI, according to Chandrasekaran. However, current AWS users will not be forced to migrate to the platform if they prefer using tools such as Redshift as standalones.

From a competitive standpoint, AWS is not the first cloud hyperscaler to combine previously disparate data management, analytics and AI capabilities.

Microsoft did so with Fabric in November 2023. Google Cloud, however, despite integrating its analytics and AI capabilities with BigQuery, has yet to unveil a fully unified platform for data management and analytics.

Data platform vendors such as Databricks and Snowflake have also blended AI development and data management over the past two years, but they lack analytics capabilities, according to Menninger.

"The AWS portfolio is one of the broadest across this spectrum," he said.

Henschen likewise noted that Fabric resembles SageMaker. Meanwhile, as more vendors provide unified environments for previously disparate capabilities, enterprises will have to choose whether to use one vendor for all their data and AI needs, or mix and match services.

"Either way, I think market consolidation lies ahead," he said.

Next steps

With the three main components of SageMaker -- Lakehouse, Catalog and Unified Studio -- now generally available, AWS plans to add capabilities such as data streaming, BI and search analytics to Unified Studio, according to Chandrasekaran.

Henschen and Menninger each noted that AWS has previously publicized its plans for those capabilities. However, how successfully they are integrated remains to be seen.

"I'll be eager to see how those integrations play out," Henschen said.

Menninger, meanwhile, added that agentic AI is still relatively new, so AWS and its peers need to develop capabilities that enable enterprises to successfully build and deploy agents.

"There's still more work to be done by AWS and others on agents," he said.

Eric Avidon is a senior news writer for Informa TechTarget and a journalist with more than 25 years of experience. He covers analytics and data management.