SAP chief AI officer: Waiting on AI is the wrong strategy

SAP is making a full organizational commitment to AI.

The German enterprise systems heavyweight has a new unit dedicated to AI development and adoption across SAP's portfolio; in January, it appointed Philipp Herzig as the company's first chief AI officer. Herzig, who has been with the company for almost 15 years, leads the new department and reports directly to CEO Christian Klein.

SAP's new AI unit is focused on driving AI development across the company, integrating AI into SAP applications and guiding customer implementation. This is at the heart of what SAP calls Business AI, which is intended to deliver business- or industry-specific outcomes through Joule, SAP's generative AI assistant.

Philipp Herzig

In this Q&A, Herzig talks about SAP's new AI-focused strategy and provides enterprises with some bottom-line advice: Waiting to experiment with AI is the wrong strategy.

Editor's note: This interview has been edited for clarity and conciseness.

In general terms, what is SAP's current strategy for AI and generative AI?

Philipp Herzig: AI is not a new topic for SAP, starting with the first stage in 2014-15. As we learn new things and face new realities, and as technology changes, we've pivoted the strategy over the years. The strategy in the new setup remains the same as it was, which is all about embedding AI in the business applications, whether it's finance, supply chain, HR. It's about getting AI into our products and business processes to help customers achieve more with less, but also to change the user experience with Joule. There's also SAP Business Technology Platform [BTP] that enables customers to build their own [capabilities] because they want slightly different versions of what we thought of or want to build their own things because they're not on the standard product.

How is the AI organization that you lead within SAP structured?

Herzig: Given that this is a huge transformation that changes quickly, we needed to take a 360-degree perspective to look at this from all those internal perspectives -- go to market, commercial, legal, marketing, services -- as we adopt internally things from SAP and other vendors. There's a dedicated team for AI marketing, a dedicated team for AI go to market, there's a dedicated team for what we call regional implementation, so the first few customers can adopt AI functions quickly as we release new things. We have cross-product and partner management functions for AI, where we look at both how our roadmap looks with what the partners want to build and how that goes together. Also, this new mandate and setup reports directly to [CEO] Christian Klein, which provides emphasis on the topic throughout all the various functions and board areas of SAP.

Can you explain SAP's concept of Business AI?

Herzig: Let's take three examples. First, there's the user experience with our AI assistant Joule. This is different because [users are] able to tap into all areas of their business at the same time. We didn't build Joule for SuccessFactors or Joule for S/4HANA public cloud -- it's one Joule. Why is that important? Let's assume you're a hiring manager for a company. You're working on HR-related functions in SuccessFactors and you use Joule to perform some tasks like changing an employee's location or opening a job requisition. But once you're done with that task, you might need to review the budget, so you just want to continue working without having to change the app. Because the finance system is connected, it can answer this financial question within SuccessFactors. Then, if you want to take a deeper dive, you can navigate into the financial applications and continue working there. We blend the boundaries and the integration points between all those systems.

Second is the middle layer. With the integration work we did, we can serve customers end to end. It's not about just making the seller more efficient in sales or a service person in field service management. It's end to end -- like recruit to retire, design to operate or source to pay, RFP writing or guided buying [in SAP Ariba] -- you can serve all of that with GenAI.

Finally, there's the underlying foundation. We built this for ourselves because we benefit from the scale, but we know that every customer wants a slightly different version of it, like a slightly different job description or RFP generation. This is where they can go on BTP into the generative AI hub, where they will see the extension points and make changes. They can't change everything, for security reasons and some of the attack vectors, but we give them dedicated points where they can make it specific to their company. Also, if you want to build a custom application because it's not in the standard application like SuccessFactors, you can build it on BTP because it's already integrated. ... We did all this heavy lifting and provide it for customers so if they [want to build an] application, they start at a higher level of abstraction to build generative AI apps much faster.

What's the relationship between advances in AI services and the cloud in SAP's strategy? Do customers need to be in the cloud to take advantage?

Herzig: We need to differentiate two cases. One is the embedded case, just using it out of the box. There, we clearly said AI and cloud-only go together. We have 27,000 customers now using AI on a regular basis, and the reality is that less than 1% of these customers are on-premises customers. So when it comes to AI, the customers are making their cloud decision already. Why is that? Because it turns out it is hard to do on premises. We can train in SAP everything we want to do in the lab, but if you come to an on-premises system where the data model has been changed or the data distribution and columns are different, all of a sudden you're facing a project. Let's say the AI algorithm promises $1 million in savings. If first you need a project that costs you $3 million [to realize those savings], it's game over.

We designed the embedded scenarios to be out of the box and as a service because we believe the adoption will happen only if you can turn it on this way. If you can reap the benefit immediately, then adoption will also happen.

However, with BTP and the AI Foundation [toolkit], we also offer customers the opportunity to build custom applications against their legacy system. There's no technical restriction. So if you want to spend the extra effort because you have a great idea that can maybe save you millions, we can help with that. But it's clear: That is a project and not an out-of-the-box experience on the application layer.

How does regulation such as the incoming EU AI Act affect SAP's AI strategy and how customers will begin using AI for business use cases?

Herzig: That comes back to the embedded nature. Let's say you want to build something that's a high-risk application according to the EU AI Act, such as for HR. Companies may refrain from it because they may be cautious of the extra effort that this entails. But we welcome the EU AI Act because we have been doing it for years.

In 2018, we published the first version of the SAP AI ethics policy, which has evolved throughout the years. This AI ethics policy ensures we do three important things. One, we state our values. Technology changes all the time, but the values shouldn't change. These values include avoiding biases and discriminatory language, as well as human oversight where the human is the final decision-maker, and [providing sources] for whatever the AI generates or recommends.

Second, we take these values and apply them from an early idea stage to deployment, because there are different concerns at design time and runtime.

Some companies have not even started and are waiting, but they should just get started and do some experiments to see what adds value in which departments.

Third is how we do it. We have a thorough AI governance body, and all the use cases that we have delivered or are on the roadmap for this year go through this governance body. The body includes SAP's chief security officer, chief diversity and inclusion officer, chief data privacy officer [and] general counsel. They look at every use case, and they can pull back ideas because they don't apply to the stated values.

When the EU AI Act came out, the reality was it didn't change anything -- we had it already.

Is AI's effect on the same scale as the internet revolution? And how should companies prepare for what's to come?

Herzig: Yes, AI is as disruptive as the internet and mobile, but the introduction or the ready adoption has happened much faster. Why is that? Because we have cloud and mobile, we have all this infrastructure. Yes, we currently have a shortage in GPUs and so on, but that's a temporary issue that will go away one day.

It's so rapid, but companies can't and shouldn't ignore it. This is my primary advice. Some companies have not even started and are waiting, but they should just get started and do some experiments to see what adds value in which departments. And of course, others are faster movers and are already doing that, but I can only really recommend if companies haven't started yet, waiting is not the right strategy.

But is there danger in moving too fast and jumping in without the proper preparation and guardrails?

Herzig: Absolutely. You should also work with a strong partner in order to prepare you and also use some of the technologies. But there are no reasons anymore from a data privacy or from a security perspective to not get started -- these things are in place. We did it this way at SAP, for example. Now, we have the generative AI hub, but we gave an early version to our employees, where they could use the [large language models] LLMs, whether they are from Anthropic, OpenAI, Aleph Alpha or Llama 2. More than 50,000 SAP employees used it last year, and we've generated more than 2.5 million prompts in various domains [such as] services, sales [and] marketing. We ensured how we can secure it and deal with confidentiality of a company, and we reaped amazing benefits from that in sales and so on. So this base level of choosing a technology and getting started with it and learning is a state that's definitely already there.

Jim O'Donnell is a senior news writer for TechTarget Editorial who covers ERP and other enterprise applications.