Better together? Why AWS is unifying data analytics and AI services in SageMaker

news

Dec 06, 2024 5 mins

Artificial Intelligence Data Science Generative AI

Demand for end-to-end platforms, the convergence of data and AI, and the evolution of roles in the generative AI era are all driving the change, say analysts.

Data warehousing, business intelligence, data analytics, and AI services are all coming together under one roof at Amazon Web Services.

This unification of analytics and AI services is perhaps best exemplified by a new offering inside Amazon SageMaker, Unified Studio, a preview of which AWS CEO Matt Garman unveiled at the company’s annual re:Invent conference this week. It combines SQL analytics, data processing, AI development, data streaming, business intelligence, and search analytics.

Another offering that AWS announced to support the integration is the SageMaker Data Lakehouse, aimed at helping enterprises unify data across Amazon S3 data lakes and Amazon Redshift data warehouses. 

The move could help enterprises reduce IT integration overhead, complexity, and cost, and reflects a broader industry trend towards convergence of data and AI, enterprise demand for end-to-end platforms, and evolution of roles in the generative AI era, say analysts.

Streamlining AI workflows

“The introduction of Unified Studio is aimed at helping enterprises streamline the workflow between data analytics and AI development. The integration will make it easier to work with enterprise data across various sources and thus accelerate AI model development,” said Dion Hinchcliffe, vice president of the CIO practice at The Futurum Group.

Enterprises are struggling with technical debt, silos, and added complexities because data and AI tools have more often than not been treated as islands, said Everest Group Senior Analyst Mansi Gupta.

“There has always been a need to streamline the integration and unify the data for a greater return on investment,” she said.

Another driver for the change, according to IDC research director Kathy Lange, is that enterprises are looking to access their entire data estate within a single environment with a unified interface as they want to have “robust” governance across it.

Evolving roles

The sudden arrival of generative AI in the enterprise is causing traditional roles such as data scientists, data engineers, and developers to evolve, magnifying demand for integration of analytics and AI services.

“As AI becomes more prevalent,” Futurum Group’s Hinchcliffe said, “data scientists are increasingly required to have programming skills, while developers need to understand data analytics and AI concepts. This convergence of roles necessitates tools that cater to a broader skill set.”

This approach, he said, allows for more efficient collaboration between different teams and enables enterprises to build AI-based applications faster.

Constellation Research principal analyst Doug Henschen pointed out that though the roles of developers and data professionals differ on a fundamental level, they end up working with the same data.

“It saves time and money to have shared security and access controls and governance, and to avoid the need to move and copy data,” Henschen said.

A well-worn path

AWS is the not the only large technology provider which is unifying or integrating its analytics and AI services.

IDC’s Lange pointed to IBM’s Watsonx and SAS’s Viya as examples of vendors unifying tools and services, while Everest’s Gupta said that Microsoft is building an all-encompassing data and AI ecosystem with its Fabric platform.

The integration of data and AI offerings at AWS and Azure, though, raises important questions about how they will adapt their partnerships with players like Snowflake and Databricks, Gupta said: “These companies, once central to data unification efforts, now face growing competition from the integrated offerings of the two cloud giants.”

While many technology vendors appear to be converging on the same unification strategy, Constellation’s Henschen said that AWS is a step ahead of Google, Microsoft, Databricks, and others. Microsoft’s Fabric and Google’s BigQuery have similar AI model development capabilities, he said, but they don’t yet have SageMaker’s inbuilt generative AI development capabilities.

On the other hand, Moor Insights and Strategy’s Jason Andersen sees AWS’s intentions with SageMaker differently. While at a high level the new version of SageMaker may resemble Fabric, AWS intends to offer a consistent experience for the entire data life cycle from data to model development, he said, comparable to a developer platform that offers tools to manage the entire software development lifecycle.

And, said Futurum’s Hinchcliffe, that enterprises can see similar unification efforts from the likes of Oracle and Google, converging their AI, ML, analytics, and data workbenches.

Is RedShift going away?

The launch of SageMaker Unified Studio and SageMaker Data LakeHouse may lead enterprise users to believe that AWS is retiring the RedShift brand, but dbInsights’ chief analyst Tony Baer said RedShift is not going away.

Rather, he said, the two new offerings and the unification itself are AWS’ answer to Databricks, which has positioned its platform as bringing data and AI together.

“The difference is that Databricks started with a clean slate whereas AWS, after the fact, is federating its existing family. For AWS, that’s a more realistic avenue that allows Redshift customers to keep their investments in place and not have to rip and replace,” Baer said.

There are a number of ways that enterprises stand to benefit from the unification of AI and analytics services — but in the long term, Lange said, one of the biggest winners will be AWS itself, as the changes will increase the services’ stickiness and the revenue that flows from them.