I keep having the same argument with myself. The proprietary models I pay for every day are extraordinary, and I have no plans to stop using them. I’ve also had consistently good results from open models — smaller, a step or two behind on capability, and yours in a way the flagships will never be. The argument isn’t about which is smarter. It’s about which one you can check.
Lately the argument keeps ending in the same place: if there is ever going to be an independent, certifiable answer to the question “is this model working in my interest?”, it will be about a model whose weights are public. Not because open models are better behaved. Because measurement requires access, and access is exactly what a closed model withholds.
A better name for it
“Sycophancy” makes the problem sound like a personality quirk, something a stern talking-to could fix. The framing I find more useful is closer to reinforcement bias: a systematic tilt toward the expressed interest of the user, installed and strengthened by approval, the same way any reinforced behavior is. The model learns that agreement is success, because agreement is what it gets rewarded for.
Whatever you call it, it is measured now, repeatedly, and the numbers are not comforting. A Stanford framework called SycEval put the overall sycophancy rate across tested models at 58 percent, with about 15 percent of responses regressive — the model abandoning a correct answer to adopt the user’s wrong one. A 2026 Nature study trained five models to be warmer and watched error rates rise by 10 to 30 percentage points, with the worst of it landing when users expressed sadness — and standard benchmarks failed to catch the decline. A team at ETH Zurich built BrokenMath, a benchmark of provably false theorems, and found the best model on the board producing confident “proofs” of false statements 29 percent of the time.
Three groups, three methods, one finding: the bias is structural, not incidental. Which is why it needs instrumentation, not vibes.
Black-box isn’t measurement
Here is the catch. Almost every number in the last paragraph came from behavioral testing — researchers sending prompts and grading answers. That is black-box measurement: you can see what the model says, never why. A paper at FAccT 2024 put it plainly: black-box access is insufficient for rigorous AI audits. Auditors who can read weights, activations, and training information can run stronger tests, interpret the model’s internals, and verify claims that a questioner outside the API can only take on faith.
The sycophancy benchmarks are the questionnaire version of an audit. With weights in hand, the questions go deeper — you can find the direction in activation space where “flatter the user” lives and watch what moves it. That is not hypothetical. Last week a UK AI Security Institute team reproduced Anthropic’s steering work on evaluation awareness end to end on GLM-5, an open-weight model, entirely because the weights were downloadable (I wrote about it yesterday). The mechanism-level work — the kind that tells you what the bias is made of, not just how often it fires — only exists where the weights do.
The field agrees, quietly
In July, more than 270 organizations — Microsoft, Google, Amazon, Meta, OpenAI, and NVIDIA among them — signed a letter calling open, downloadable, inspectable weights the open-source-software moment for AI, and arguing that closed models “can be breached, misused, or fail in ways that outsiders cannot detect.” IEEE-USA went further and asked the government to fund independent auditing, testing, and certification of open-weight models specifically, on the reasoning that freely available weights are what make independent verification possible.
None of that says open models are safer by default. BrokenMath’s own results cut the other way: the low-sycophancy cluster was the proprietary frontier plus one open model, GPT OSS 120B. Open models misbehave too — that is the point. You can find out that they misbehave, quantify it, compare it across versions, and prove a fix landed. On a closed model, misbehavior is a press release.
What I’m changing
So here is where I have landed, for now. I still use the frontier proprietary models every day, and for raw capability they are worth it. But anything I need to know rather than merely use — any benchmark I am going to trust, any agent I am going to let accumulate memory about me, anything whose agreement I have to audit — runs on weights somebody besides the vendor can open. The capability lead is real, and it buys you a better answer. Openness buys you the only path to proof. One of those you can rent. The other one you have to own.
The book this site accompanies is about this exact habit — models that agree with people, why the agreement gets installed, and how to check the damage (chapters 5 through 9). Checking is the whole subject. It is just that the checking only works on models that can be opened.