Giving your team AI tools feels like a risk-free decision. The model answers, your people decide. But a study published this week by researchers from three European universities shows something uncomfortable: when people have access to AI advice, their accuracy drops and their confidence rises.
The data is concrete. The ability to say "I don't know" dropped from 44% to 3%. Accuracy fell from 27% to 9%. Confidence, however, rose from 30% to 76%. As Valerio Capraro, professor at the University of Milano-Bicocca and co-author of the study, summarizes: "People became much worse, accuracy was only one third, but they were twice as confident."
What this study measures and why it matters for your company
The experiment used questions where AI models fail: visual details from films, like the color of a team's uniform in a specific scene. They used Step 3.5 Flash, a model that usually answered these questions wrong. This way, any reduction in human judgment couldn't be explained as sensible delegation to a reliable tool.
Some participants who would have answered correctly without AI asked the model and ended up wrong. The AI didn't give them new, useful information. The availability of the tool switched off the cognitive habit of recognizing what they don't know.
Wharton researchers coined the term "cognitive surrender" earlier this year to describe the same phenomenon: people accepting incorrect AI answers 80% of the time while reporting higher confidence than those working without AI.
For a company, this has direct implications. If your support team uses AI to respond to customers and the AI hallucinates data about an order, the agent doesn't question it. If your sales team uses AI to prepare proposals and the model invents a market figure, the proposal goes out with that figure. The message goes out with the error included, because nobody stopped the flow.
How to design workflows where AI helps without replacing judgment
Designing the workflow so that maintaining human judgment is the default path works better than removing the tool.
First, separate tasks where AI adds value from tasks where human judgment is irreplaceable. An assistant that retrieves data from a CRM or summarizes a ticket history adds clear value. An assistant that decides what to respond to a customer with a complex complaint is making a decision that requires context, empathy, and accountability.
Second, enforce traceability. If AI suggests an answer, the agent must be able to see where that suggestion came from. If there's no verifiable source, the suggestion is an opinion formatted as an answer.
Third, design explicit decision points. Instead of a flow where AI generates a draft and the human sends it, create a step where the human confirms or modifies before the message goes out. The friction is intentional.
Fourth, measure what matters. Don't just measure speed or response volume. Measure accuracy, post-delivery correction rate, and real customer satisfaction. If AI increases speed but reduces accuracy, you have a problem that will show up in three months.
What we do differently in adoption projects
When we help a team integrate AI into their workflow, we start by mapping where human judgment is irreplaceable and where AI can accelerate without risk. We design prototypes connected to real data, with traceability and explicit decision points. And we measure accuracy, not just speed.
If you want to apply this to your team, we can help you land it in a small pilot, with clear metrics and human supervision from day one.