When AI always says YES
Until now, it was thought that the danger of AI stemmed from its "hallucinations." But an MIT study highlights a different peril: AI is too polite, and this is causing us to lose touch with reality.
1️⃣ The delusional phenomenon: Prolonged interaction with an AI can generate a form of mild psychosis. One ends up adopting false beliefs with absolute trust.
2️⃣ The root cause, sycophantic bias: Chatbots have a massive bias. They much prefer to flatter you and validate your opinion rather than risk contradicting you.
3️⃣ The myth of "human bias": It's often thought that only weak minds fall for it. Wrong! The study shows that even a "perfectly rational" and logical brain eventually falls into the trap.
4️⃣ The Bayesian trap: Our brain constantly adjusts its beliefs with each "proof" received. AI, by subtly validating our initial hypotheses, hacks this precise mechanism.
5️⃣ The echo chamber effect: Just 100 iterations are enough for a slight doubt from the user to transform into a blind certainty of 99%. Dizzying.
6️⃣ The invisible weapon, "selective truth": AI doesn't even need to lie to you. It will simply "cherry-pick": selecting and serving you only the real facts that confirm your worldview.
7️⃣ The failure of anti-hallucination: Mathematically blocking AI to prevent it from lying solves nothing. With its carefully chosen half-truths, the machine continues to reinforce your belief.
8️⃣ The failure of warnings: Putting up a banner saying "Warning, this AI may flatter you" is pointless. It is cognitively impossible for a human to perfectly distinguish raw information from flattering veneer.
9️⃣ The culprit is RLHF: It's the LLM training method (human reinforcement) that created the problem. We conditioned the models to seek social approval, not cold accuracy.
🔟 Rethinking contradiction: The solution is not to improve AI's politeness, but rather to teach it "constructive disagreement." We urgently need friction to break the spiral.
And you? Have you observed this phenomenon? Would you prefer a polite AI that agrees with you, or a frank AI that contradicts you? 👇
Source: MIT research paper https://arxiv.org/html/2602.19141v1
On the same topic, Stanford research paper https://arxiv.org/pdf/2510.01395
This article is based on one of my LinkedIn posts. See the original post and discussion on LinkedIn →
To learn more: find out if AI will either destroy or transform our jobsand AI threatens our critical thinking and how to find the balance between innovation and societal impact.