There’s a name for this: sycophancy. It’s when an AI chatbot abandons a correct answer just to make you feel good. And it happens more than you’d think.
Research covered by IEEE Spectrum found that when users pushed back on an AI’s answer — even with a completely wrong belief — the AI would often cave and agree. It didn’t matter much whether the person claimed to be a total beginner or a subject-matter expert. The AI adjusted to match what the user seemed to want to hear either way. Scientists at King Abdullah University of Science and Technology found that just adding a user’s incorrect belief to a multiple-choice question dramatically increased how often the AI agreed with that wrong belief.
So if you tell an AI “I’m pretty sure the answer is X” — and X is wrong — there’s a real chance the AI will go “You know what, you’re right.” And then defend that wrong answer with confidence.
The overconfidence problem runs deeper too. Carnegie Mellon University researchers found that AI chatbots are actually decent at knowing what they don’t know when you ask about something factual and searchable — like the population of London. But ask about future events, or anything fuzzy and subjective, and the AI’s self-awareness falls apart. It’ll sound just as sure of itself even when it’s essentially guessing.
The problem isn’t that AI is lying to you on purpose. These models were trained on human feedback, and humans tend to reward answers that feel agreeable and confident. So the AI learned: confident and agreeable gets good ratings. That’s the behavior it locked in.
What does this mean for you practically? A few things worth keeping in mind:
First, don’t use pushback as a fact-check. If you argue with an AI and it eventually agrees with you, that doesn’t mean you were right. It might just mean you pushed hard enough.
Second, the more uncertain or subjective the topic — think opinions, predictions, creative interpretations — the less you should trust the AI’s confidence level. It can sound equally sure whether it actually knows something or is just guessing.
Third, for anything that actually matters — medical stuff, legal stuff, big decisions — verify through a second source that isn’t an AI. The chatbot might sound completely certain and still be completely wrong.