← Read the book

Chapter 1 · Part I · Read in full

Give Disagreement Somewhere to Go

Give disagreement somewhere to go.

Read in full7 minute readBy Timothy O’Brien

Selected excerpt. Opening through “Wow, you made such a great decision” from The World That Agrees With You.

Generative AI can help us research, write, calculate, build, and understand problems that would otherwise take much longer to work through. Those benefits are a reason to use the tools carefully. An assistant that produces useful work can also reinforce a mistaken assumption, overlook an inconvenient fact, or bring an old misunderstanding into a new conversation.

This book began with the wish to have chosen well. That wish makes reassurance valuable to a seller, a news organization, and an assistant. The systems around us have become increasingly capable of discovering what we respond to, including what makes us angry and whom we distrust. Generative AI adds the ability to compose a response for the person asking. Memory lets earlier responses and interpretations influence what comes next.

The result can be helpful, flattering, or both. The practical question is whether you can find out what an answer rests on and change it when the evidence changes. A response that makes you feel understood has not, by that feeling alone, established that its judgment is sound.

Make correction part of the work

Begin with one decision that matters. Give the assistant identifiable evidence and a specific task. Ask it to separate documented facts, your reports, and its interpretations. Have it perform calculations with tools you can inspect. When seeking another assessment, keep the evidence fixed and avoid telling the reviewer which conclusion you want.

Check these steps in the actual output. A shortened report should keep its important qualifications. Correcting a memory should prevent the old conclusion from influencing another answer. Explaining how much you want to proceed should not turn a failed test into a successful one. You can inspect those results instead of relying on an assistant's assurance that it is being objective.

You do not have to investigate every suggestion. Match the effort to the consequences. A dinner recommendation can tolerate uncertainty that a performance review, financial proposal, or engineering decision cannot. Separate contexts when information belongs to different purposes, and make time to review memories that will affect future answers.

For an agent or an application, build these habits into the system. Show the context selected, preserve sources, control memory writes, and check a proposed action before executing it. A person should be able to pause recall, correct a record, and understand what clearing history actually removes.

Someone must be able to act

Personal care reaches its limit when another organization uses AI to make a decision about you. You may not choose the service, inspect its prompts, or control its memory. If a mistaken conclusion affects your application, your work, or your access to a service, you need somewhere to take the correction.

An institution using these systems should name the person or office that can reconsider a consequential decision. Explain how an affected person can submit evidence, when they can expect a response, and who can change the outcome. A general feedback form does little if the message reaches nobody with that authority.

Fund the review. Give the reviewer time, access to relevant evidence, and permission to question the original decision. Keep a record of the grounds for the decision and of corrections that require earlier work to be revisited. Someone must be responsible for finding the people and decisions affected by an error.

That work can be uncomfortable. A manager may have recommended the system and promised a saving. An inconvenient finding can mean delaying a rollout, reopening cases, or admitting that the benefit was smaller than expected. Agree on the tests and the grounds for stopping before those costs arrive. The wish to have chosen well operates in institutions too.

If this book agrees with you

By now, you may have thought of several people who ought to read this book. That is an easy response to a book about sycophancy. It is also a flattering one: other people need correction, while you have recognized the problem.

Apply the same scrutiny here. The studies support particular findings. The broader connection between personalized information, agreeable assistants, memory, and our willingness to reconsider is an argument for you to assess. It should remain open to evidence that these systems help people recognize uncertainty, repair misunderstandings, and act more reliably on correction.

Take that question to a decision of your own. What evidence would change your mind? Who could bring it to you? Would they have the time and permission to do so? You can continue using these tools with confidence in their usefulness while requiring better grounds for their conclusions.

Give disagreement somewhere to go. When a correction is justified, change the answer, repair the memory, and revisit the decision that depended on it.