SAP CTO Philipp Herzig Says Building an App Is Not the Same as Running a Business

Generating is not running

AI coding tools can spin up an app in minutes, and many people have taken that as proof that traditional software is finished. Philipp Herzig, chief technology officer at SAP, told The Rundown AI that this take confuses building an app with running one, and that the same gap explains why so many AI pilots never reach production.

Generating an expense app is one thing, Herzig said. Running it across 40 countries, keeping up with tax rules, enforcing permissions, and reconciling reimbursements is another. As agents take on more autonomous work, they depend more on trusted data, business context, governance, and security. The interface can change radically. The need for reliable business processes does not.

Adoption is the harder problem

Herzig says the distance between a prototype and a production ready system at enterprise scale is still substantial human work. Asked whether technology or adoption has been harder at SAP, he answered adoption, without a doubt. Useful AI changes who makes which decisions, and that affects expertise, authority, and professional identity.

His adoption advice is practical enough for a small team. The companies moving fastest gave access first to people whose judgment the organization already trusted. A respected supply chain manager saying the tool helped catch a disruption persuades colleagues better than any training program, because the proof comes from a person, not a vendor.

Fluent but wrong is the real risk

The sharpest idea in the interview is a phrase every team using AI should borrow. Herzig calls the riskiest answers fluent but wrong: confident, correctly worded, and blind to how the business actually runs.

His reasoning is that company data lives in tables, and knowing what a field means requires the business process behind it, including unwritten rules, exceptions, and identity, like knowing a supplier in procurement is the same entity in finance. The SAP answer is a Knowledge Graph that maps what company data means, plus a Company Memory that stores how past decisions got made, above all why an exception was approved.

One caution comes with it. An exception must not become a rule just because it worked once. A human process owner still reviews why the decision worked, whether the conditions still apply, and whether it is safe to generalize.

Stop sending everything to the biggest model

Token prices keep falling, but company AI bills keep rising as usage spreads. Herzig says SAP learned it cannot blindly throw everything at large language models. An email summary does not need the same model as complex business reasoning, so tasks should be routed by complexity, risk, and context, or solved without a model call at all.

He names two wastes: sending everything to frontier models, and context bloat, meaning prompts stuffed with low value information. More than 60,000 SAP colleagues use its Joule Work system internally, so those choices are tested at production scale rather than in a demo.

What a small team can steal

Herzig runs technology for one of the largest software companies on earth, but most of his advice scales down to a studio, a shop, or a one person business rolling out agents this month:

• Pick one high value use case and give it first to the person the team already trusts
• Test agents with inputs your real process would reject, like an invoice with no currency, and see what they accept
• Route simple work to smaller models and save the expensive ones for real reasoning
• Keep a record of why exceptions were approved, not just that they were approved

The responsibility stays human

The interface is changing from clicking screens to stating intent. The responsibility is not changing at all. Someone still owns whether the output is correct, secure, and safe to act on, and that ownership is the job AI is creating inside every business that adopts it.