The LLM Council: how five AI advisors beat one confident answer

Ask a large language model a genuinely hard question, "should we cut our hero product, reprice the range, or hold the line?", and it will answer instantly, fluently and with total confidence. That confidence is the trap. What you received is one perspective, dressed as the perspective. The model picked a plausible lane, committed to it, and never showed you the three other defensible positions it quietly drove past. For a low-stakes query that is fine. For a decision with real money or reputation attached, a single confident answer is exactly the wrong tool.

The fix is not a better prompt. It is a better process. The LLM Council, a method popularised by Andrej Karpathy, treats one question as a brief for a panel rather than a lookup. Instead of trusting a single response, you convene several advisors, let them work the problem independently, have them critique each other anonymously, and then appoint a chairman to weigh the debate and deliver a verdict. The same underlying intelligence, run as a council instead of an oracle, produces noticeably sharper thinking.

§01Where the method comes from

The idea traces to Andrej Karpathy, a founding member of OpenAI and former director of AI at Tesla, who built a small project he called LLM Council. His insight was that different models, asked the same question, disagree in useful ways, and that the disagreement itself is signal. So he wired up a workflow: send a prompt to several leading models, have each one review and rank the others’ answers without knowing who wrote what, then let a final "chairman" model read the whole debate and compose the definitive response.

The anonymity is the clever part. Stripped of brand names and egos, each answer is judged on its merits alone, which surfaces the strongest argument rather than the most famous source. The method borrows from how good human institutions already reach hard decisions, juries, peer review, investment committees, and ports it onto AI. You do not need five different models to feel the benefit, either: a single capable model, instructed to genuinely inhabit five distinct roles, captures most of the value on its own.

“The idea traces to Andrej Karpathy, a founding member of OpenAI and former director of AI at Tesla, who built a small project he called LLM Council.”

§02The five advisors, and why each one earns a seat

A council only works if the seats are genuinely different. Five clones nodding along is theatre. The point is structured disagreement, each advisor holds a distinct mandate and is told not to hedge, so the question gets stress-tested from angles a single answer would smooth over.

• The Contrarian hunts for failure modes and weak assumptions, the constructive skeptic whose job is to find what breaks before reality does.
• The First-Principles Thinker refuses the question as framed and rebuilds it from fundamentals, often revealing you were solving the wrong problem.
• The Expansionist ignores the downside for a moment to map the upside everyone else is too cautious to name.
• The Outsider brings deliberately naive, domain-free eyes, the perspective that asks the "obvious" question no expert dares to.
• The Executor cares only about feasibility, what actually ships, what the first concrete step is, and whether the plan survives contact with Monday morning.

§03How a council run actually unfolds

The process is deliberately staged so that no single voice dominates. It starts with framing, not answering. The question is restated neutrally, with its real stakes and constraints made explicit, because a council fed a sloppy brief produces a confident, sloppy verdict. Relevant context is pulled in up front so every advisor argues from the same facts.

Then the five advisors respond simultaneously and independently, each in their own lane, each forbidden from fence-sitting. Next comes the round that makes the method work: anonymous peer review. Every advisor reads all five responses, now labelled only A to E, and identifies the strongest argument, the biggest blind spot and the gap the group collectively missed. Finally a chairman synthesises everything into a structured verdict, where the council agrees, where it genuinely splits, what everyone overlooked, a clear recommendation, and the single next action to take.

§04When to convene a council, and when not to

The council is a heavy instrument, and using it on everything is its own kind of mistake. It earns its cost only when the question has stakes, real uncertainty, and more than one defensible answer. Spend it where being wrong is expensive and a fresh pair of eyes genuinely changes the outcome.

• Convene it for: strategic pivots, pricing and positioning calls, "build versus buy", a make-or-break campaign bet, or any choice you keep going back and forth on.
• Skip it for: factual lookups, simple yes/no questions, creative drafting, or anything with a single correct answer, a council there is just slower and more expensive theatre.

§05What this means for a D2C brand

Marketing decisions are exactly the shape the council was built for: high stakes, real ambiguity, several plausible moves and a strong pull toward whichever option you thought of first. Should you chase a new audience or deepen the one you own? Cut the discount that is bought you growth but eroded your margin? Pour the budget into the channel that is working, or hedge into the one that might? These are not lookup questions, and a single confident AI answer on any of them is a liability dressed as a shortcut.

Run them through a council instead and the first-thought bias gets challenged by the Contrarian, the framing gets rebuilt by the First-Principles Thinker, the missed upside gets named by the Expansionist, and the whole thing gets pressure-tested for feasibility before a dollar is committed. You still decide. But you decide having seen the disagreement, not just the most confident summary, and that is the entire difference between using AI as an oracle and using it as a room full of advisors.

§06Decisions are where marketing is won

Most of the money in marketing is made or lost at the fork in the road, the pricing call, the positioning bet, the channel reallocation, long before any ad goes live. The brands that compound are not the ones with the most confident answers; they are the ones that interrogate the important decisions hardest. A council is simply a disciplined way to do that with the AI you already have.

This is the same instinct we bring to the brands we work with: treat the big calls as something to pressure-test from every angle, not to settle on first impulse. The intelligence is cheap now. The discipline to use it well is the advantage.