The artificial intelligence (AI) market is expanding rapidly. Enterprise AI spending surged eight-fold in 2024 according to the Menlo Ventures. This year is expected to be a banner year for AI adoption. However, many AI initiatives are not delivering measurable returns.
Recent AI studies paint a stark picture. MIT reports that 95% of generative AI pilots fail to reach production, while RAND found that 46% of companies scrapped their AI projects before launch. Perhaps most telling, Boston Consulting Group discovered that only 24% of firms have developed the capabilities needed to move AI beyond proof-of-concepts into tangible business value. While we could debate what the actual percentages are, one thing is clear. Companies are spending money on AI but aren't getting the returns they expected.
What separates the AI strategy winners from the also-rans? It appears the key to AI success is selecting a set of narrowly scoped use cases with a quantifiable cost that will benefit from using AI. As Best Buy said, we don't have AI projects, we have projects that use AI. SAP's journey offers one success story with crucial lessons for any organization struggling to prove AI's worth.
Some of the best AI success stories start by addressing a simple problem. For example, Cisco discussed how leveraging AI to detect network configuration issues solved a key customer support challenge.
For SAP, the path to AI success began with a simple but expensive problem: customer support for its over 300 million cloud subscribers. With an average cost per support case of estimated €198, the German enterprise software vendor was hemorrhaging money on avoidable service requests. Customers were struggling to find answers across SAP's fragmented knowledge base, leading to unnecessary support tickets that could have been resolved through self-service.
"There's no centralized search system in the SAP suite," explains Michelle Lewis-Miller, Michelle Lewis-Miller, the Head of Digital Experience and Head of Voice of Customer for SAP Customer Support. "We weren't hitting our self-service targets. We couldn't see what was happening and couldn't measure anything.”
Visibility is a key issue across businesses of all sizes. Many companies deploy AI solutions without clear metrics or understanding of where the technology can create the most impact. SAP started with a measurable business problem and worked backward to find the right AI solution.
Before implementing any search solution, SAP had already invested heavily in knowledge quality. Lewis-Miller uses a culinary metaphor to explain: "If you're serving dinner with poor ingredients, it doesn't matter how great the chef is."
Too many organizations rush to implement AI technology without first ensuring their underlying data and content are ready to support it. Knowledge wasn’t the company’s problem. SAP had over 10 million knowledge assets—spanning help documents, knowledge base articles, community posts, and videos- as a foundation for its search efforts. "SAP had already made massive investments to get the knowledge into good shape. The problem was the search," Lewis-Miller notes.
Rather than launching a company-wide AI initiative, SAP began with a targeted pilot in their Concur travel and expense division. The Concur division was already using Coveo's AI-powered search platform, making it an ideal testing ground for generative AI capabilities.
The Coveo AI pilot results exceeded all expectations. Within six months, SAP Concur experienced a 30% drop in support case volume, translating to €8 million in reduced annual costs. For many customers, this was their first encounter with generative AI—and it delivered immediate, tangible value.
"When we piloted Relevance Generative Answering in Concur, it went so well it was shocking," Lewis-Miller recalls. "We saw such a huge decrease in case submission that it changed our whole budget for the next year."
The Coveo-Concur pilot’s success illustrates a key principle: AI ROI comes from solving specific, measurable problems rather than implementing technology for technology's sake. The Concur team had clear metrics (case volume reduction), a defined user base, and a focused use case.
The cultural shift at SAP proved crucial to the initiative's success. In a discussion with Lewis-Miller, she emphasizes that executive support went beyond budget approval: "Our executives came in and made a very clear statement. They said, You're not going to be punished for trying." This created an environment where teams could be decisive rather than seeking consensus from dozens of stakeholders.
She refers to this as the "tip of the spear approach," noting that "at an organization of any size you need to find the people who are willing to take risks, and you need to have the executive leadership being unequivocal about supporting that person who's willing to take the risk." Without this top-down support for calculated risk-taking, the project likely would have stalled in committee discussions and consensus-building exercises that ultimately prevent any meaningful progress.
Emboldened by the Concur results, SAP faced a much larger challenge: implementing AI-powered search across SAP for Me, the central portal serving millions of customers. Unlike the focused Concur deployment, this required integrating 14 to 20 different knowledge bases while serving vastly different user types—from certified engineers with 30 years of SAP experience to small business owners with limited technical background.
The complexity initially threw the team. When they deployed the same generative AI technology on SAP for Me, the results looked worse at first glance—higher click rates and more case submissions seemed to indicate the system wasn't working.
"We expected it to address the low-hanging fruit. It wasn't at all like what we saw with Concur. We asked Coveo if something was broken in their backend," Lewis-Miller admits.
The breakthrough came when SAP's analysts realized they were measuring the wrong things. While simple questions decreased on the Concur platform, SAP for Me users were asking much more complex queries. The AI wasn't deflecting low-value cases—it was helping resolve sophisticated problems that would have required expensive expert support.
"We were deflecting a more valuable subset of cases," Lewis-Miller explains. "When we changed our perspective and looked at the overall numbers, we saw that submissions had actually gone down."
Here's where many AI projects fail: teams panic when initial metrics don't match expectations and abandon promising initiatives. SAP took a different approach, partnering with their technology vendor to dig deeper into the analytics. Successful AI implementations require a company to deploy, monitor, analyze and iterate. SAP's story highlights how theory rarely matches reality.
Since launching in 2023, the AI-powered search has deflected 1.6 million support cases, generating €186 million in annual cost savings. The technology now spans 47 different sources and indexes 11.2 million documents across SAP's ecosystem.
SAP's success stems from choosing the right technical approach for their specific needs. Rather than building custom AI models from scratch, which is labor and time intensive, SAP leveraged a "hybrid search" system that combines multiple AI techniques:
This hybrid approach addresses common search frustrations: lexical systems miss relevant results when you don't use the exact right keywords, while pure generative AI can sometimes provide inaccurate information. The combination delivers both precision and intelligence.
SAP's €186 million in first year savings didn't happen by accident. Behind those impressive numbers lies a methodical approach that any organization can follow—if they're willing to resist the allure of flashy AI demos and focus on fundamentals. Key steps that every organization can take include:
While all of these points are important, Lopez Research has seen many companies skip the governance and compliance step. Lewis-Miller identifies meticulous documentation as a critical but often overlooked success factor. "We invested a lot of time in finding employees who had the soft skills necessary to interpret the technical detail for the legal people and the legal detail back to the technical detail." This documentation became essential reference material that prevented the project from stalling at multiple compliance checkpoints. At enterprise scale, having detailed outlines for every architectural and legal decision proves invaluable.
SAP isn't stopping at cost reduction. The company is now using behavioral analytics to intervene before customers encounter problems. Lewis-Miller describes the vision: "As a user is engaging in one of our support journeys, we can intervene in real time to get ahead of issues before they actually emerge."
The approach involves using customer behavior patterns to identify when someone is struggling to find information, then proactively routing them to chat support before frustration sets in. "We're not trying to give everybody a white glove experience. We're specifically intervening only in those moments where we can see it's going down the wrong path," Lewis-Miller explains.
This includes predicting system issues before they occur and routing insights to product teams to fix confusing features before they generate support cases. By analyzing search patterns and support interactions, SAP can identify friction points in their products and address them proactively—turning support data into product improvement fuel.
The AI revolution is about implementing the right technology to solve real business challenges. SAP found their formula. Now it's time for other enterprises to find theirs.