400,000 Customers, 20,000 APIs: SAP's CTO on the Hard Exam of Enterprise AI

Sarah Guo opens No Priors with SAP CTO Philipp Herzig by stating a fact a surprising number of engineers still miss: everyone knows the name, few can describe the breadth. Herzig's version is simple. SAP is the market leader in enterprise applications and platforms, with 400,000 customers running finance, HR, supply chain, manufacturing execution, logistics, warehouse management, plus sales, service, commerce, and procurement. He does not call it a suite so much as the operating system of a company — the system that takes a business from order to cash and source to pay, end to end, around the world.

Guo, an early-stage investor, wants to know how that franchise survived internet, cloud, mobile, social, and now AI, when each wave was supposed to crown a new guard. SAP is still, she notes, the largest-market-cap enterprise software vendor, outlasting a generation of challengers. Herzig has heard the obituaries. Last year the market argued about an AI bubble; this year the slogan is that SaaS is dead. The cost of building software has collapsed with generative coding, he grants, but something older has kept winning.

That something dates to 1972. The founders, still at IBM, implemented the same finance system at customer after customer and decided the economics did not scale. A person can only add so much value in a given day, and the programs were almost the same except for a thin layer of local specifics. Standard software was born from that observation: write it once, sell it many times. The stack moved from mainframe to client-server to internet to mobile, and the product changed with each wave. What customers wanted did not. They wanted outcomes, a return on investment, and a way to get the work done.

The same logic, Herzig says, is why SAP is all-in on AI. Gartner draws two races, an AI innovation race and an AI outcome race, and the gap between them is widening rather than closing. Enterprise adoption is not where he wants it. His job is to close that gap on the foundation SAP already has, while re-engineering the system itself. Internally that starts with generative coding: every developer, in every language, is using it as a productivity boost. Externally it starts with products that cut time and cost on work customers already pay for.

Joule for Consulting is the example he reaches for first. It is one of SAP's fastest-growing AI products, built for consultants who have to move large, heritage-heavy landscapes into the cloud and onto new AI features. Herzig says it can take about 30 percent of the effort out of that work, which is time and money. Travel-booking and expense agents are live on Concur. From a CTO's chair, though, the deeper change is the same one the industry lived through when on-prem software moved to the cloud. At first people thought lifting the old product onto the internet *was* cloud. Then they had to learn CI/CD, daily deploys, and how to scale multi-tenancy up and down. The software got rebuilt. AI, he argues, is doing that again, on three levels.

The first level is the interface. Classical software assumed the intelligence sat in the chair: you designed a UI to teach a human how to finish a task by clicking through screens. That time is over. SAP calls the replacement generative UI — screens assembled on the fly for the question at hand. Ask what new tariffs, taxes, or tension in the Strait of Hormuz mean for your supply chain, and the system should analyze it against SAP data, not against a generic chatbot. The product also becomes proactive. It can run overnight and greet someone in the morning with a sales-order drop, a supply-chain snag, and a short list of recommended next steps.

The second level is the business process. Order-to-cash used to be a relatively rigid standard operating procedure. Agents, Herzig says, can blend the structured and unstructured worlds so more of the work actually gets done — the shift some people call service-as-software or outcome-as-a-service. Underneath that sits the data layer. SAP already holds a company's most valuable records: general ledger, invoices, warehouse, inventory. Those have to be joined with the rest of a firm's information into one harmonized semantic view, because, as he puts it, AI is only as powerful as the data. That globally harmonized model is what he wants to put under the agents.

Asked for the biggest engineering challenge, Herzig does not name a model. The hard part is teaching AI to do the right thing at scale. Two years ago everyone built a RAG demo: a chatbot on ten documents that could blow a CEO's socks off. A hundred documents got harder. A thousand became a real engineering problem. Inside Joule the same question has to return different answers for different people. A U.S. SAP employee asking about travel policy should not hear the German answer. The assistant has to know country, payroll, and tax — it has to be wired to master data, not just to a pile of PDFs.

MCP repeats the pattern. Last year anyone could stand up a server and hook up ten APIs; at a hundred, context windows start to blow. SAP has 20,000 APIs. The problem is no longer whether tool-calling is possible. It is how to offer an integrated experience so a finance question, an HR question, and a supply-chain question can be asked in the same conversation and actually correlate. Scale, he says, is the thing entrepreneurs keep underestimating.

Guo asks whether agents inside these processes will drive the UI the way a person would, or call APIs. Herzig does not claim a final answer. Computer use already looks phenomenal, but it is still slow, and most of the work, he thinks, will live in tool calling, with agents running in the background — nobody wants a browser open all day. Computer use still matters where APIs are missing or a legacy system has to be bridged for a while. The domains that pay off first are the unstructured ones language models already handle well: services, support, sales, and document-heavy knowledge work. Joule for Consulting was an early win for that reason. Structured work came later. Terms like "order" are overloaded — maintenance, sales, and purchase orders are different objects — so the models had to get better at tools, and SAP had to build orchestrators that disambiguate. Chat-with-your-data is the payoff: instead of waiting on an analyst's dashboard that is good enough 80 percent of the time, a user can talk to the system, pin the view, manage the problem for a few weeks, and delete the tile when it is gone. The glue that finally unlocked tables in finance, sales, and supply chain is SAP's knowledge graph, sitting between natural language and structured data.

The development lesson Herzig has been repeating on his own circuit is that people have to start writing evals. Agentic coding works, he tells Guo, because you can verify the outcome: does the program compile, do the unit tests pass. Taste and a lot of hard engineering from Anthropic and OpenAI produced strong code models on top of that loop. Finance is not the same. If you want a reliable output, you need data that says: given this input, that is the output, so the agent can assert against it. The mindset shift is learning to describe boundary conditions to the harness — security, data privacy, and code quality, because the code still has to be maintained on day two, three, and four, not just generated as a first draft.

He laughs at the flashback. As a computer-science student he sat in a lecture where Google engineers said they could leave at 5 p.m. because they had written their tests. Test-first, test-driven development: it is coming back. Almost nobody did it then, and he includes himself. Writing the code first was more fun, and product managers handed over messy requirements, so you discovered how the system should behave while you built it. Now the writing of code is largely automated. What remains is describing the outcome you actually want. That is a different job for developers, especially as the models have taken more of the steps since last December.

Guo wonders whether enterprise agents can compound the way code-generation models have, given that business intent is not as obviously verifiable. Herzig splits the problem into two lanes. The system of record already gives you a starting condition: you can see in the database that if order-to-cash runs this way, accounts receivable should land that way, with these taxes. That is verifiability, and it is not enough for the autonomous-enterprise vision. Too much of the knowledge is tribal — in people's heads, Slack, Teams, or a phone call, nowhere in the system. Agents will come back with clarifying questions, the way Claude Code and Codex still do, and those answers have to be stored. SAP used to call the capture of how work actually runs process mining. It now calls it agent mining: record the decision traces, then either flag an anomaly against the standard operating procedure or promote a better local practice, say from Australia, into a global one. Every trace, every user input, every bit of agent observability becomes new data, which becomes new evals, which lets the system optimize toward a verified outcome. That is the flywheel.

Herzig's least fashionable view is that predictive and tabular models still matter, and that LLMs are not the whole answer. Companies plan by forecasting: seasonal demand in a retail store, production volume in a plant, cash flow in finance. Cash flow itself breaks into familiar machine-learning problems — will the customer pay on time (classification), and if not, how long is the delay (regression). Language models generate one token after another. They are excellent at language. They are not made for these predictors. The fallback is still XGBoost, AutoGluon, and other AutoML stacks, and that path does not scale. You still have to hire strong data scientists and run a classical project: data, features, model selection, country by country.

He has a concrete scar. At a pharmaceutical company operating in 90 countries, just predicting payment delay meant two models per country — 180 models to train. SAP's bet was to copy what large language models did for unstructured data onto the structured world sitting in hundreds of thousands of tables. Give the system a little context and a small amount of data, not a huge labeled set, and get high-accuracy predictions. Two years of research, published at NeurIPS and other conferences, produced RPT-1: relational pre-trained transformers. It is still a transformer, with a very different architecture, covering classification, regression, and sometimes time series. Herzig thinks it will let far more people make the predictions language models struggle with.

That research sits next to the adoption gap Guo asked about — innovation versus outcomes. The primary blocker, Herzig says, is data. Most companies have disaggregated information because of old purchasing choices or M&A, and that fragmentation caps what AI can do. Customers who did the data-integration homework over the last ten or fifteen years have a much easier time. Scale is the second blocker: the bigger and more complex the landscape, the harder it is to present a unified experience. Security is the third. OpenClaw, he says, is full of ideas, but no CISO is going to deploy whatever is on GitHub into the company. A recent LiteLLM vulnerability that could steal keys and credentials is the kind of incident that ends a security chief's job. Amazing demos do not ship until those three problems are treated as first-class.

If the transformation works, finance, HR, and supply-chain teams will not spend their days collecting information and building PowerPoints. Decisions get faster and better; mundane work is automated; people run more scenarios and go deeper, which Herzig calls more strategic thinking. He compares a finance shared-service employee to a junior developer with Claude Code. The person is no longer tasked with writing most of the code. They supervise, give feedback, capture what the result should look like, review, and think about what to build next. Every role, every level, gets upleveled as today's work is pushed down to agents. People achieve more because a lot of intelligence is baked into the system, including the parts of the job that were never much fun.

That shift hits SAP's own business model. Most of the software is still seat-licensed, with exceptions such as Concur, Fieldglass, and the business network. Herzig says it was always clear that AI would move the company, step by step, into a consumptive world, and then, once verifiability is high enough, toward outcome-based licensing of the kind Sierra is exploring. Today the commercial offer is hybrid: consumption underneath, with a remaining seat element, because customers are not ready for a purely metered bill. They want cost predictability. They do not fully trust every outcome yet. They worry usage charges could explode. Some buyers are already leaning forward; others still demand the classical model. The transition is a joint journey, and nobody knows how fast it will run.

Looking five years out, Guo asks what will separate winners from losers among incumbent software companies, rather than a universal markdown of every SaaS multiple. Herzig's answer is adoption and the outcome you put in front of the customer. For most companies, he says, the technology does not matter. He tells SAP developers that their job is to make the technology disappear. Architecture stays flexible: partner with all the model vendors, invest only in what is differentiating, and skip layers that will be commoditized. Enterprise-grade security and integration have to be there so customers can turn capabilities on almost instantly. If value takes too long to show up, the return on investment evaporates and the business case gets harder. That, not the fashion of the stack, is what he thinks will sort the field.

On a personal close, Guo asks how a CTO of SAP actually spends the day. Most of it is reviewing progress with teams across every layer, from the database to the models to the UI — giving guidance, learning, watching what happens outside the company. He also prototypes constantly. While they are talking, he has a bunch of command-line sessions running, trying things, seeing what works, and feeding that back to the team as inspiration.