Consultants' AI Blindspot: 95% Accuracy Uncovered, But at What Cost?

SAP conducted an internal experiment to gauge consultant attitudes toward AI, which revealed striking results. Five teams were asked to validate answers to more than 1,000 business requirements completed by SAP's AI co-pilot, Joule for Consultants, a workload that would normally take several weeks. Four teams were told the analysis had been completed by junior interns fresh out of school, and they reviewed the material, finding it impressive, and rated the work about 95% accurate. The fifth team, however, was told the very same answers had come from AI, and they rejected almost everything. According to Guillermo B. Vazquez Mendez, chief architect of RI business transformation and architecture at SAP America Inc., "the lesson learned here is that we need to be very cautious as we introduce AI, especially in how we communicate with senior consultants about its possibilities and how to integrate it into their workflows." Mendez noted that the consultants' initial rejection of the AI-generated answers highlights the importance of effective communication and understanding of AI capabilities. When the fifth team was asked to validate each answer one by one, they discovered that the AI was, in fact, highly accurate, surfacing detailed insights they had initially dismissed. The overall accuracy remained about 95%. The experiment was conducted to understand how consultants perceive AI-generated work and how it can be integrated into their workflows. The results suggest that consultants may be skeptical of AI-generated work, especially if they are not informed about the capabilities and limitations of the technology. This finding has significant implications for the adoption and implementation of AI in various industries, including consulting and professional services. The use of AI in consulting and professional services is becoming increasingly prevalent, with many companies leveraging AI-powered tools to automate routine tasks and provide insights to clients. However, the results of SAP's experiment highlight the need for effective communication and understanding of AI capabilities to ensure successful adoption and integration of AI into workflows. The experiment has sparked a wider discussion about the role of AI in the consulting industry and the importance of effective communication and understanding of AI capabilities. As AI continues to evolve and become more prevalent in various industries, it is essential to address the challenges and opportunities associated with its adoption.