Last month, I was fortunate to be invited to present at an SAP AI summit event. It was a great event and extremely gratifying to see so much energy and lively discussion around topics like SAP Business AI, SAP BTP, and SAP Joule. However, throughout the conference one thing stood out—there was not much content out there for customers still running legacy ECC or Business Suite systems.
This content gap led to many interesting conversations with companies that have been left wondering how they can tap into the latest AI innovations without completely overhauling their SAP landscape. In my mind, this is a gap that needs filling, because there are definitely ways to bring AI into legacy environments and drive real value.
With all that in mind, I thought I would put together an article highlighting some practical recipes for integrating the latest and greatest AI innovations into legacy SAP systems. These approaches are designed to help you tap into the power of AI without the need for a full system overhaul, making it easier to modernize and stay competitive while maximizing the value of your existing investments. Bon appétit.
1. AI-Powered Copilots / Agents
AI-powered copilots are all the rage these days. In the SAP world, there's SAP Joule - the AI-powered assistant baked into SAP cloud solutions like SAP SuccessFactors and S/4 HANA Cloud, Public Edition. Outside of SAP, other leading vendors like Salesforce and Workday have introduced similar solutions in recent months. And, of course, Microsoft was the first on the scene with Microsoft Copilot which we covered at length here.
With a couple of exceptions, SAP Joule was primarily designed to work for customers running SAP SaaS solutions. So, if you're like most SAP customers that aren't on that list, then a great option for building/deploying copilot solutions is Microsoft Copilot and Copilot Studio. With Copilot Studio, we can build copilot solutions that integrate with SAP and other related knowledge sources/business systems to create highly-productive user experiences that deliver context-aware insights, routine task automation, and streamlined workflows—all tailored to the specific needs of your business.
SAP as the Brain for Generative AI
The key to a successful chatbot or copilot lies in its knowledge base, and with SAP connectivity, we have access to all the rich business data we need. With Microsoft's SAP ERP and OData connectors, we can surface data from backend SAP systems of all kinds (no matter how old) and analyze/synthesize it using the generative AI features baked into Copilot Studio (see Figure 1 below).
Figure 1: Integrating SAP with Microsoft Copilot and Copilot Studio
This kind of SAP integration enables the copilots we develop to respond to queries with up-to-the-minute accuracy, whether it’s a question about an invoice status, stock availability, or a production schedule. By embedding this knowledge into the chatbot’s intelligence, we're able to create an AI-powered assistant that isn’t just conversational but also contextually aware of specific business processes.
Automating Repetitive Tasks
In addition to its abilities to use SAP as a knowledge source, Copilot Studio also supports the creation of callable actions that can be used to incorporate automated SAP workflows into agent experiences. These action definitions point to SAP APIs (e.g., RFCs or OData services) but—more than that—they describe what a particular SAP API can do so that the generative AI engine within Microsoft Copilot knows which action to trigger in response to natural language prompts from the user.
Overall, these capabilities enable us to design chatbots or agents that don’t stop at answering questions—they can take action. To put this concept into perspective, check out the customer service agent shown in Figure 2 below. Here, a customer service rep is able to ask questions about assigned service orders in SAP, inquire about statuses, and so forth. Behind the scenes, Copilot is converting these requests into action calls to fetch live data from the SAP backend.
Figure 2: Custom SAP Customer Service Agent Running in Microsoft Teams
Of course, the agent experience isn't just limited to information coming from SAP. We can also pull in data from external knowledge bases, scheduling systems, and even customer service playbooks to provide service reps with a fully integrated experience. We also have the option of using Adaptive Cards technology to build lists and forms like the one shown in Figure 3 below. In this scenario, the customer service rep can see information about an assigned work order and then apply status updates which flow back to SAP in real time.
Figure 3: Enhancing the Copilot UX with Adaptive Cards
Bridging the Gap Between People and Processes
Our ultimate goal with intelligent agents is to build a bridge between your workforce and core business processes. These AI-driven experiences can engage employees and customers alike, delivering insights and executing tasks in ways that are faster, smarter, and more intuitive than traditional workflows. The result? A leaner, more agile organization that fully harnesses the power of its SAP data.
With deep SAP integration, your chatbot and copilot experiences can move beyond scripted interactions to deliver generative AI solutions that feel dynamic, proactive, and deeply integrated into your business operations. We'll have much more to say about these types of SAP-powered agent experiences in the coming weeks.
2. Autonomous Agents
As copilot/agent experiences have evolved, a new class of agents has emerged: autonomous agents. Autonomous agents blend traditional workflow solutions with generative AI, making it possible for workflows to think critically, adapt to changing inputs, and operate more independently. For complex business processes—like the ones running in SAP—this kind of fusion is a game-changer, boosting efficiency and sparking innovation across the entire organization.
Microsoft recently announced a slew of new autonomous agent capabilities at Microsoft Ignite 2024. To see how these autonomous agents work, check out the video below.
From an SAP integration perspective, we can utilize the same approach we demonstrated previously in Copilot Studio to create interactive agents. This unlocks some powerful use cases such as:
Responding to an email sent to a shared inbox by reading through the email body and/or attachments, determining the intent from the sender (e.g., a customer), and automatically posting transactions to SAP.
Listening for updates made in SAP (e.g., a status update) in real-time and dynamically determining how to relay that update to interested stakeholders.
Dynamically routing incoming requests to the right individuals (e.g., the user responsible for a service order).
The Copilot and Agent Weave
While copilots and autonomous agents are powerful enough in their own right, it can be particularly interesting to weave them together into longer-running process flows. This concept is demonstrated in Figure 4 below.
Figure 4: Copilot and Autonomous Agent Weave Concept
Following along with the process flow contained in Figure 4, you can see how control is passing back-and-forth between autonomous agents running in the background and copilot sessions running in the foreground:
The process is initiated by a service technician that has put an SAP service order on hold due to the fact that a replacement part is unavailable.
Behind the scenes, this status update triggers an autonomous agent which is able to analyze the order, interpret the meaning of the user status, and initiate a search for a replacement part in SAP.
After scanning through inventory and sourcing records, the autonomous agent may have determined multiple sourcing options. At this point, we can use the triggering capabilities of Copilot Studio to "wake up" a copilot on the frontend to prompt an inventory specialist to make a selection.
The inventory specialist confirms their selection, for example, within Microsoft Teams, and the interactive agent seamlessly fades into the background.
Finally, an autonomous agent picks up where the previous one left off and is contextually-aware of the part selection, availability date, and even crew schedules. It can then use this data to schedule a follow-up visit within SAP and/or dispatch systems.
As discussed in a previous blog post, all these new capabilities are completely revolutionizing user experiences, enabling more intuitive interactions, faster decision-making, and seamless integration across systems. For SAP users, this shift figures to fundamentally transform the way users engage with system (and maybe finally spell the end of SAP GUI-based experiences).
3. ChatGPT Integration
Looking beyond copilots, another useful generative AI-based recipe involves integration with ChatGPT and services like the Azure OpenAI service. Here, we can take data from SAP—including both structured and unstructured data—and use it to ask questions in ChatGPT just like you would in the foreground using the ChatGPT website.
In the scenario shown in Figure 5 below, you can see where a Summarize button was added to a transaction in SAP. Here, whenever the user presses the button, all the information about the selected order—including its attachment files—is packaged up and sent to the OpenAI service to generate a quick summary of the order. Users could then review the summary online or forward it as an email to interested parties.
Figure 5: Integrating ChatGPT Functionality into SAP Transactions
While this sort of solution does require some custom development, it's actually pretty low-hanging fruit in the grand scheme of things. So, if you're pining for some of the shiny generative AI features built into S/4 HANA Cloud, this recipe can be used to gain a quick win and score some major points with users.
4. AI-Powered Search
Search is another area where we can really inject some major AI goodness into SAP. Using services like Azure AI Search, we can easily build sophisticated, AI-driven search indexes that make finding objects in SAP as intuitive as running through a Google search. These search experiences can be seamlessly woven into SAP GUI transactions, Fiori apps, or even legacy Web Dynpro apps.
From Keywords to Contextual Intelligence
To put this AI-powered search concept into perspective, consider a scenario we developed for a customer that was looking to streamline material lookups. Like many SAP customers, they had enhanced the material database and defined a complex set of characteristics as well. Rather than having to define an endless array of search helps, their goal was to have one simple search help that would support all manner of search queries (see Figure 6).
Figure 6: Accessing AI-Powered Search from a Search Help in the SAP GUI
In order to power a search experience like the one shown in Figure 6, we needed to build a search index in Azure AI Search. From a development perspective, this mostly amounted to the development of a custom ABAP report program to extract material data and write out a CSV file. Of course, if you're already replicating this data to say a cloud data warehouse like Snowflake, Google Big Query, or Microsoft Fabric, then this part is already done, so you're ahead of the game.
Once the data is loaded into something like Azure Blog storage, the Azure AI Search service makes it easy to import the data and build/train/test the search index. This is mostly a configuration exercise and highly scalable (see Figure 7).
Figure 7: Building a Search Index with Azure AI Search
Over time, the search index naturally becomes smarter as we continue to feed it with more master data and incorporate feedback loops from API calls.
A Game-Changer for Operational Efficiency
While it's easy to dismiss user-friendly search helps as a "nice-to-have", it's worth noting that in our material scenario users were spending a considerable amount of time each day trying to hunt down the right material. What's more, there were notable consequences if they ended up selecting the wrong material for certain processes.
At the end of the day, incorporating AI-powered search experiences into SAP isn’t just about creating better lookups—it’s about transforming how employees interact with critical business systems. With AI-powered search, your core business processes becomes faster, smarter, and more intuitive, ensuring that your SAP system remains a tool for innovation rather than a bottleneck.
5. Document Intelligence
Earlier this year, we demonstrated how Microsoft's low-code AI Builder tool could be used integrate several different types of AI models into SAP business processes. Between the built-in document processing capabilities of AI Builder and its more industrial strength cousin, the Azure AI Document Intelligence service, it's surprisingly easy to train models to extract data from Word and PDF documents or even image files.
These AI models can then be easily incorporated into larger-scale workflows such as the one shown in Figure 8 below. In this scenario, we're using Power Automate, Microsoft's cloud-based workflow automation service, to look for files uploaded to SharePoint and OneDrive. The Power Automate workflow process(es) then call out to AI Builder to extract data from the documents and then post that data to SAP.
Figure 8: Low-Code Document Intelligence Solution Concept
This type of AI integration is perfect for automating routine processes in accounting, HR, and other business areas. By harnessing AI to intelligently process invoices, employee records, and similar documents, these solutions eliminate the need for tedious manual data entry, improve accuracy, and dramatically speed up processing times.
It's also worth noting that these approaches can be quite a bit cheaper than spot solutions you might purchase that only handle the invoice verification process, for example. Plus, this kind of streamlined approach allows you to achieve automation without requiring extensive custom development or the disruption of overhauling existing systems.
6. Computer Vision
Looking beyond document intelligence, there are many other practical uses for AI-powered computer vision services. For example, imagine an SAP mobile inspection app where inspectors need to fill in a lot of inspection data. Rather than keying in all this data by hand, picture this: an inspector simply snaps a picture of the equipment using their mobile device and the system pre-fills the inspection form automagically.
We were able to implement this exact scenario at a customer using Azure AI Document Intelligence. Here, we trained a computer vision model on various equipment nameplate images and then used the extracted data to power a lookup into SAP to pull up additional equipment information. For the inspectors, the experience was seamless: they simply snapped a picture of the nameplate, and within seconds, the app automatically pre-filled their forms—saving time and eliminating manual data entry.
Figure 9: Using Computer Vision to Scan an Equipment Nameplate
Besides information extraction, computer vision can also be used to help users classify objects or identify anomalies. For example, for an insurance company, we might use computer vision to analyze attachments collected in SAP to verify claims or detect potential fraud scenarios.
Zooming out a bit, computer vision can also be used to build AI-powered knowledge bases. This concept is illustrated in Figure 10 below. Here, we can use document cracking techniques to extract data from attachment files, photos, or even images of handwritten notes to supplement the structured data captured in SAP transactions. This data can then be fed into AI-powered search services or directly into copilots to enrich user experiences and drive continuous improvement.
Figure 10: Using Document Cracking to Build an AI-Powered Knowledge Base
7. AI-Powered Forecasting
The last recipe we'll look at involves the use of AI to build various types of forecasts. Even if you're on the oldest of legacy ECC systems, it's fairly easy to extract data out of your SAP system database and feed it into pre-built AI models in tools like Azure Machine Learning. Refer to Figure 11 to see how this can all come together.
Figure 11: Using Azure Machine Learning to Build a Forecasting Model
Once we've trained the AI model, we can then easily feed it into visualization tools like Power BI to review the forecast. Or, we can feed the results back to SAP (e.g., as planned requirements).
Another interesting wrinkle here is to use the AI model to refine parameters in SAP. For example, we might use the model results to adjust inventory-related fields on the material master such as safety stock, lead times, and so forth.
Standing on the Shoulders of AI Giants
While Figure 11 might make the process look effortless, training AI forecasting models like this is far from simple. It requires a careful and collaborative effort between data scientists, who understand the technical nuances of AI and machine learning, and business process experts, who provide crucial context about the workflows and data involved. Together, they must ensure that the model accurately interprets the data while aligning with the specific needs of the business, a process that demands precision, iteration, and deep domain knowledge.
However, with that said, it's worth noting that cloud platforms like Azure host thousands of AI models and there are plenty of templates/examples to bootstrap the process. Depending on your exact scenario, it's very possible that you could adapt one of these templates pretty easily to match up with your data and get up and running pretty quickly.
Closing Thoughts
The recipes we covered in this article are really just the tip of the iceberg when it comes to the AI-powered features you can bring to your legacy SAP systems. From chatbots and search to document intelligence and beyond, the possibilities for enhancing processes, reducing manual effort, and driving smarter decisions are nearly limitless. AI opens the door to transforming SAP into a modern, agile system without needing a complete overhaul.
Of course, making these integrations work takes collaboration between technical teams and business experts. But with the right mix of skills and tools, there's absolutely nothing stopping you from using AI technology to breathe new life into your SAP systems.
