AI Scoring System Exposed: 95% Accuracy, Then Reality
- What: SAP conducted an internal experiment to assess consultant attitudes towards its AI co-pilot, Joule for Consultants.
- Why it Matters: Highlights the challenge of integrating AI into professional workflows, particularly overcoming initial skepticism from experienced professionals.
- What's Next: SAP is focusing on communicating the benefits of AI as a tool to augment, not replace, consultant expertise, aiming for the "consultant of 2030."
“`html
SAP‘s Internal Experiment Reveals Consultant skepticism Towards AI
Table of Contents
The Experiment: Accuracy Judged by Perception
SAP ran an internal experiment to gauge consultant attitudes toward AI. Five teams were tasked with validating answers to over 1,000 business requirements generated by Joule for Consultants, a task that typically takes weeks. Four teams were led to believe the analysis was performed by junior interns. They found the work impressive,rating its accuracy around 95%.The fifth team, though, was informed the answers came from AI.
This team overwhelmingly rejected the AI-generated responses. Only through individual validation of each answer did they realize the AI’s accuracy matched the intern-attributed work – approximately 95%.
The Lesson: Interaction and Integration are Key
Guillermo B. Vazquez Mendez, chief architect at SAP America Inc.,emphasizes the importance of careful communication when introducing AI. the experiment underscored the need to address potential biases and clearly articulate AI’s role as an enabler, not a replacement, for consultant expertise.
This finding has become a central point in SAP’s strategy for developing the “consultant of 2030” – a professional who leverages AI to focus on higher-value tasks and strategic business insights.
Overcoming AI Skepticism: Amplifying Expertise
Vazquez notes that resistance to AI is understandable, particularly from consultants with extensive experience and institutional knowledge. However, AI copilots like Joule are designed to amplify expertise, not supplant it.
“What Joule really does is make their very expensive time far more effective,” Vazquez says. “It removes the clerical work, so they can focus on turning out high-quality answers in a fraction of the time.”
The core message is consistent: AI is a tool requiring human oversight, freeing consultants from tedious documentation searches and allowing them to concentrate on delivering detailed, impactful solutions.
The Consultant Time-Shift: From Technical Execution to Business Insight
Historically, consultants have spent approximately 80% of their time on technical aspects – understanding systems, data flows, and function execution.conversely, clients dedicate 80% of their time to business-focused activities.
This disconnect is where AI, specifically Joule, aims to bridge the gap.
“There’s a gap there - and the bridge is AI,” Vazquez says. “It flips the time equation, enabling consultants to invest more of their energy in understanding the customer’s industry and business challenges.”
Impact on Consultant Roles: A Data-Driven View
The shift in time allocation can be visualized as follows: