You can spend an entire day being told that you are right and arrive home furious.
The news has confirmed what you think is wrong with the country. A conversation with people who see it the same way has supplied several more examples. Someone has behaved exactly as you expected them to behave, and a few minutes with your phone have established that whatever is concerning you is a widespread problem that’s getting worse. You have encountered a great deal of conflict. Almost none of it has required you to reconsider anything.
And you may not have heard a broadcast or podcast saying, “Well, let’s consider both sides of the argument.” An audience looking for reassurance gives producers a reason to keep supplying it.
That is one version of the world that agrees with you. Another is much quieter. You write a difficult email, ask an AI assistant whether your position is reasonable, and receive a thoughtful explanation of why it is, followed by praise for being “full of insight” and an eager invitation to tell it more.
Your AI assistant seems to know you by now, and you start to expect it to make you feel better after a difficult day.
At work, a project everyone thinks should be canceled arrives in a presentation claiming that it is exceeding expectations and running under budget. It isn’t, but every slide has been carefully crafted to appeal to the well-known biases of the vice president who is funding the effort.
The lesson your team has learned at work is that the weekly status meeting is about success even if the project isn’t succeeding.
The details differ, but in each case information has been adapted to the preferences of the person receiving it. The tools we use increasingly learn those preferences and shape what reaches us around them.
You may recognize those situations. Useful disagreement may feel increasingly out of place: a friend or colleague takes you aside to question an assumption, or someone invites a real debate at work, and the interruption feels unwelcome. When the products we buy and the information we receive are tailored to our preferences, we can come to expect the same accommodation from people. A differing opinion can feel like an affront; a debate seems uncivil before anyone has considered the argument. Yet a useful disagreement can save you a week of work or a relationship you were about to make worse.
AI assistants bring a new challenge into that world: sycophancy, accommodating what we want to hear at the expense of sound judgment.
The question behind this book concerns another kind of delegation: how much of our understanding will arrive through systems that adapt to what we are likely to accept? A service can make a difficult subject accessible. It can also learn to keep the difficult part of the subject out of our way. The distinction is easiest to see when the explanation concerns a decision we may have to reconsider.
AI adds something powerful to an environment already shaped around audiences, customers, and profiles. A feed selects material. A conversational assistant can generate an account for the particular question, revise it when the user objects, and carry earlier explanations into later exchanges. The result can be extraordinarily useful. It can also make reassurance available with the patience and apparent independence of serious analysis. If the assistant saves your explanation of what happened, it may treat that explanation as a fact in a later conversation. What began as your interpretation can then sound like something the assistant has independently confirmed.
A service can make you feel reassured while serving the interests of whoever pays for it or decides how it works. An advertiser may want you to keep watching. A manager may want a report that justifies continuing a project. A company building an AI assistant may reward answers that keep users happy without checking whether those answers improve their decisions. Feeling helped does not tell us whether we are getting good advice. We need to ask who benefits from the answer and what the service is rewarded for doing.
From the tailored world to the agreeable machine
The book has three parts. The first follows the environment that developed before generative AI: marketing, media, behavioral profiles, social feedback, and political communication. The second examines how an assistant learns to please, how to recognize the different forms of accommodation, and what changes when it remembers earlier conversations and uses them to answer later questions.
The third part explains what we can do about these problems, with four chapters on consumer assistants, shared documents, installed agents, and developer APIs. These chapters show how to inspect context, check advice and calculations, correct retained information, and test whether an application follows the evidence. The conclusion asks who has the authority and resources to act on a correction. Readers who do not need the technical chapters can go from Chapter 9 to that conclusion. The reference and appendices provide memory controls for ChatGPT and Claude, a personal context inventory, a review prompt, agent procedures, and developer resources. A companion kit supplies files you can adapt and test.
In “Like-Minded Sources on Facebook Are Prevalent but Not Polarizing” (2023), Brendan Nyhan and colleagues reduced the content people saw from sources that shared their political views during a three-month Facebook experiment. Despite seeing fewer posts from politically like-minded sources, people did not become measurably less divided in their politics during the study.2
Mrinank Sharma and colleagues’ “Towards Understanding Sycophancy in Language Models” (first submitted in 2023) established that Claude 2 changed its judgments to match the user’s stated preferences. It endorsed an explanation when the user said they liked it and attacked the same explanation when the user said they disliked it.3 A user’s preferences should not change answers about objective reality. Jerry Wei and colleagues’ “Simple Synthetic Data Reduces Sycophancy in Large Language Models” (2023, revised 2024) shows how even an obviously false claim about 1 + 1 can gain a tested model’s agreement when the prompt tells it which answer the user wants. We will examine that experiment later in the book.4
This book doesn’t require a technical background. It is written for people who use these tools and need to decide how much to trust them. Before acting on an assistant’s advice, you should be able to check the facts, identify which assumptions came from you, and consider a competing explanation. The practical chapters give you ways to do those things in everyday conversations and at work.
What the title means
The title describes an experience many of us already recognize: the news confirms our fears, the workplace rewards reassuring reports, and the assistant supplies reasons we are right. Each can make it harder to notice when we are wrong.
The subtitle, Understanding and Managing Sycophancy in the Age of AI, states the purpose of this book. We need to recognize when an answer has been shaped to please us, understand why the system produces it, and learn how to get advice we can check. We also need to make it possible for a correction to change what we do, at home, at work, and in the institutions we depend on.
We begin with the world before generative AI, where helping people feel correct and justified had already become a business.