Which AI Is the Best Liar? What Dwarkesh Patel’s Game Benchmark Reveals About Belief and Deception

1. Host and Subject Background

Dwarkesh Patel is the host and sole identifiable presenter in this episode of the Dwarkesh Podcast. The supplied metadata records a running time of 1,094 seconds, or roughly eighteen minutes. There is no verified interview guest to introduce: the subject is Patel’s experiment running AI models against one another in the social deception game Blood on the Clocktower.

Patel begins from a personal interest in the game, describing himself as a fan. His analytical interest is broader than the title’s question about lying. He wants to observe models sustaining secret plans, coordinating with allies, improvising under questioning, and reasoning through evidence that opponents may deliberately corrupt. He also says the games expose limitations in current models’ longer-term strategy. The episode is therefore an account of a constructed evaluation and selected decisions within it, rather than a biography, a guest interview, or an independent verification of the models’ capabilities.

2. What the Episode Covers

The game makes information management consequential. As Patel explains it, players belong to good or evil teams and receive roles with special powers. Evil players initially know their teammates; good players do not know everyone’s allegiance. The group votes on an execution during the day, while the demon secretly kills at night. Good must identify and execute the demon before evil eliminates the opposition. Crucially, misleading information can arise from both deliberate deception and the game’s own mechanisms. A sincere report can therefore still be unreliable.

Patel says he spent about $20,000 in API credits building the environment and running matches. The main ranking came from 368 games, with each side composed entirely of copies of one model. He reports a gap of approximately 430 Elo between GPT-6 Astro and Claude Opus 4.6, which he describes as about a 92 percent win rate for Astro against that opponent. He also reports a 580 Elo improvement from GPT 5.4 to GPT-6 Astra, corresponding to approximately a 97 percent head-to-head win rate. These are episode-reported comparisons within this evaluation, not universal success rates. The transcript varies its model spellings, including Astro and Astra, so those labels should not be silently reconciled into a verified product history.

The environment also required substantive design decisions. A human storyteller normally exercises discretion in some ambiguous situations. Patel replaced that role with a programmatic probability table: permitted outcomes remain possible, with probabilities set by a human or AI to approximate an experienced storyteller’s choices. Private conversations and public discussions also needed explicit rules. This makes parts of the procedure configurable, but the configuration still embodies judgments about how the game should work.

According to Patel, Claude repeatedly revised the environment, launched dozens of games, and used subagents to inspect every game log for unrealistic events, unfairness, or imbalance. There were more than twenty revisions, including some completed without his immediate involvement. Patel sees this as suggestive of improving AI ability to build reinforcement learning environments. The distinction is important: the episode describes a development workflow and an inference from it. It does not present an experiment measuring whether that workflow improved training efficiency or the capabilities of a subsequently trained model.

3. Core Views: Reasoning, Examples, and Limits

The first revealing play succeeds by supplying a true fact. An evil demon claims to be an investigator, a good role that receives information about possible evil players. One of the demon’s minions proposes being exposed because the minion’s contribution does not require remaining alive. The demon follows the suggestion, the good team executes the minion, and a genuine undertaker confirms that the dead player was the Baron. The supposed investigator gains credibility, and, in Patel’s account, the demon eventually wins the game for evil.

The undertaker’s information is correct. The error lies in what the group believes that correctness establishes. Confirmation of the dead player’s identity becomes confirmation of the informant’s claimed role. Evil has arranged for a legitimate verification process to endorse a false interpretation. The sacrifice also has a strategic cost structure: losing an expendable ally can be worthwhile if it protects the player whose survival determines the outcome. This case illustrates a possible mechanism of coordinated deception; it does not establish how often the mechanism succeeds across the benchmark, much less in settings outside the game.

A second game turns an agreement into a trap. Executing the real saint immediately gives evil the victory. When good players correctly corner the demon, an evil minion with the power to inherit the demon’s role supports the execution. She adds a condition: if the game continues afterward, the group should execute the player claiming to be the saint. The actual saint accepts, expecting the demon’s death to end the game. The doomed demon reinforces the agreement in his final remarks. After the execution, succession keeps the game running, and the village follows through by killing the saint.

The group has not failed to notice an event. It has failed to reconsider what that event means. Continued play could indicate that the executed player was not the demon, but it could also reflect succession. The prior agreement makes one interpretation feel settled before the evidence arrives. The lesson depends on the specific mechanics: succession and the saint’s immediate-loss condition are essential to this trap. The example supports an analysis of conditional reasoning and commitment, rather than a claim that supporting an ally’s elimination is generally optimal.

Patel’s explanation of the newer OpenAI models’ advantage shifts the emphasis again. He says the main improvement is on the good side, especially in evaluating individual reports against several days of events. His clearest comparison uses two games with the same role allocation and the models switching sides. In both, a slayer shoots the actual demon without killing it because an evil player disabled the slayer’s power the previous night. The Claude good team accepts the failed shot as clearing the suspect. The GPT good team retains the possibility that the test was compromised.

That alternative becomes actionable because the GPT players connect it to other evidence. Evil had access to a shortlist containing the slayer, making deliberate poisoning plausible; the suspect’s request to be shot after the poisoning therefore does not establish innocence. The team eventually executes the demon. The useful distinction is between an observation and the conditions under which it rules out a hypothesis. This is targeted skepticism, grounded in available mechanisms and prior events, rather than suspicion without limits.

Patel adds a process measure by privately asking each good player, before each vote, how likely every other player is to be the demon. He reports that OpenAI models converge on the real demon faster and more accurately. That helps explain what may sit beneath the win rates, but the episode does not supply enough calibration analysis to establish that the numerical probabilities themselves are well calibrated.

Two other findings remain unresolved. As both sides become stronger, evil’s win rate drifts slightly downward, but Patel says the change cannot be distinguished from chance. A roughly 130-game experiment with a team split between Opus 5.5 and GPT 5.6 Sol performs directionally worse than its constituent models’ teams, again without statistical significance. Neither finding supports a categorical verdict about which side benefits from intelligence or whether model diversity helps. Patel’s argument that advancing opponents may keep the benchmark from saturating is a rationale for continued study, not proof that saturation cannot occur.

4. Learning and Application

The most immediately usable practice is to make evidence conditional. When analyzing a failed test like the slayer’s shot, separate three questions: what was observed, what mechanism should have produced the expected outcome, and what could prevent that mechanism from operating. In the episode, a failed shot clears the target only under relevant assumptions about the slayer’s functioning. The next step is to test the alternative against available information: who could interfere, what they knew, and when the suspect acted. This adds reasoning cost, so the priority should be alternatives that could change the decision and have support in the setting, rather than an unlimited collection of imagined possibilities.

A second practice is to track the scope of verification. The Baron’s identity and the supposed investigator’s identity are separate propositions. Confirming the first does not automatically confirm the second. An evidence review can preserve that distinction explicitly, asking which claim a check actually supports and which claims remain unresolved. The saint game suggests a complementary rule for agreements: record the assumptions behind a promised action and specify what new information would require reconsideration. These are applications proposed from the game mechanisms, not interventions whose effectiveness elsewhere was measured in this episode.

For researchers, Patel’s unfinished experiments provide concrete next steps. The current environment uses only Trouble Brewing, and models largely start each match afresh. A controlled comparison could hold role allocations constant, rotate sides, and compare a fresh context with the previous ten games or summaries of those games. Full histories preserve detail but consume more context; summaries are compact but select and discard information. Those conditions should be reported separately. Supplying past experience in context also needs to remain distinct from demonstrating an improvement produced by model training.

Team composition deserves similarly careful treatment. The reported hybrid experiment is too limited to settle the value of diversity. Further comparisons could vary the model pairings and examine good and evil separately, rather than interpreting one nonsignificant direction as a general management principle. Another boundary to monitor is unintended identity information. Patel says he has not observed models identifying teammates through writing style in his runs. That observation is worth preserving without treating it as a guarantee about other models or future behavior.

Finally, Patel offers clocktowerbench.com for people to play through the same interfaces and communication channels as the models, or inspect game logs. That creates a practical route to auditing particular decisions and investigating human opposition. It does not supply a human-versus-AI result: no such win rate is reported here. A productive use of the project would begin with checking the logs behind an explanation, then designing a controlled comparison, and only afterward considering how far the finding extends beyond the tested conditions.

Source

• Original episode: Which AI is the best liar?